AOARD-TR-94-03 


AOARD  REPORT 

The  FRIEND21  Project  for  the  Development  of 
Generation  Human  Interface 


Feb  2-4  1994 
T.  Davis 
AOARD 


the  Next 


A  summary  of  the  '94  FRIEND21  International  Symposium  on  the  next  generation  human/computer 
interface,  conducted  Feb.  2-4, 1994  in  Tokyo,  Japan  is  presented,  along  with  some  background  on  the  six 
year  Ministry  of  International  Trade  and  Industry  (MITI)  sponsored  FRIEND21  Project.  Abstracts  of 
all  symposium  presentations  (except  the  keynote  address)  are  included.  The  keynote  address,  by  Dr. 
Kochiro  Tamura,  Deputy  Director-General  of  the  Electrotechnical  Laboratory,  is  included  in  its 
entirety.  Also  included  are  the  titles  of  selected  English  language  background  papers  from  earlier 
phases  of  the  FRIEND21  project.  This  report  is  prepared  jointly  with 
Dr.  David  K.  Kahaner 
US  Office  of  Naval  Research  Asia 

(From  outside  US);  23-17, 7-chome,  Roppongi,  Minato-ku,  Tokyo 
106  Japan  (From  within  US):  Unit  45002,  APO  AP  96337-0007 
Tel:  +81  3  3401-8924,  Fax:  +81  3  3403-9670 
Email:  kahaner@cs.titech.ac.jp 

based  upon  information  collected  via  symposium  attendance,  review  of  the  symposium  proceedings,  a 
post  symposium  visit  to  the  Institute  for  Personalized  Information  Environment  (the  MITI 
organization  responsible  for  the  FRIEND21  project),  review  of  selected  FRIEND21  background  papers 
from  earlier  project  phases,  and  conversations  with  other  symposium  attendees. 

DISTRIBUTION  STATEMENT  A: 

APPROVED  FOR  PUBLIC  RELEASE;  DISTRIBUTION  IS  UNLIMITED. 

AIR  FORCE  OFFICE  OF  SCIENTIFIC  RESEARCH 

ASIAN  OFFICE  OF  AEROSPACE  RESEARCH  AND  DEVELOPMENT 

TOKYO,  JAPAN 
UNIT  45002 
APO  AP  96337-0007 
DSN:  (315)229-3212 
Comm:  81-3-5410-4409 


E 


1995032]  134 


The  FRIEND21  Project  for  the  Development  of  the  Next 
Generation  Human  Interface 


Abstract 

A  summary  of  the  '94  FRIEND21  International  Symposium  on  the  next 
generation  human/computer  interface,  conducted  Feb.  2-4,  1994  in  Tokyo, 

Japan  is  presented,  along  with  some  background  on  the  six  year  Ministry  of 
International  Trade  and  Industry  (MITI)  sponsored  FRIEND21  Project. 

Abstracts  of  all  symposium  presentations  (except  the  keynote  address)  are 
included.  The  keynote  address,  by  Dr.  Kochiro  Tamura,  Deputy  Director- 
General  of  the  Electrotechnical  Laboratory,  is  included  in  its  entirety.  Also 
included  are  the  titles  of  selected  English  language  background  papers  from 
earlier  phases  of  the  FRIEND21  project.  This  report  is  prepared  Jointly  with 

Dr.  David  K.  Kahaner 

US  Office  of  Naval  Research  Asia 

(From  outside  US):  23-17,  7-chome,  Roppongi,  Minato-ku,  Tokyo 
106  Japan  (From  within  US):  Unit  45002,  APO  AP  96337-0007 
Tel:  -1-81  3  3401-8924,  Fax:  ■f81  3  3403-9670 
Email:  kahaner@cs. titech.ac.jp 

based  upon  information  collected  via  symposium  attendance,  review  of  the 
symposium  proceedings,  a  post  symposium  visit  to  the  Institute  for 
Personalized  Information  Environment  (the  MITI  organization  responsible  for 
the  FRIEND21  project),  review  of  selected  FRIEND21  background  papers  from 
earlier  project  phases,  and  conversations  with  other  symposium  attendees. 

1.  FRIEND21  Project  Background 

The  FRIEND21  Project  was  initiated  in  1988  by  the  Japanese  Ministry  of 
International  Trade  and  Industry  to  develop  next-generation 
human/computer  interface  technologies.  The  title  FRIEND21  is  derived  from 
"Future  Personal  Information  Environment  Development",  with  the  suffix  21 
indicating  that  the  activity  is  targeted  for  the  advanced  information  oriented 
society  expected  in  the  early  2D'  century.  The  six  year  project  is  scheduled  to 
conclude  on  March  31,  1994  at  the  end  of  the  Japanese  Fiscal  Year  1993. 

When  the  FRIEND21  Project  was  begun  in  1988,  its  administration  was 
delegated  by  MITI  to  the  newly  created  Institute  for  Personalized  Information 
Environment,  P.I.E.,  whose  responsibilities  include  overseeing  the  research 
activities,  conducting  related  technology  surveys  and  assessments,  and  public 
promotion  of  the  FRIEND21  Project.  According  to  P.I.E.  the  total  government 
expenditures  for  the  six  year  project  duration  are  ¥900M. 


The  P.I.E.  office  address  and  telephone  numbers  are 


17-1  Toranomon  1  Chome 

Minato-ku 

Tokyo,  105  Japan 


Tel:  -t81  3  3507  0791 
Fax:  -1-81  3  3507  0921 
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Mr.  Eiji  Kuriyama  is  the  Executive  Director,  Mr.  Kenji  Seno  is  the  General 
Manager,  Engineering  Department  and  Mr.  Daizo  Kato  is  the  Deputy  Director, 
FRIEND21  Research  Center. 

As  indicated  in  Section  2  below,  a  total  of  twenty  sponsoring  members, 
including  thirteen  electronics/computer/communications  related  companies, 
three  publishing/printing  companies  and  four  banks,  are  participants  in  the 
FRIEND21  Project.  P.I.E.  is  very  proud  of  the  fact  that  all  of  the  original  project 
sponsors  continued  their  support  for  the  entire  duration  of  the  FRIEND21 
project. 

P.I.E.  operates  the  FRIEND21  Research  Center  (located  in  Tokyo)  which  is 
staffed  by  a  mix  of  permanent  P.I.E.  researchers  (project  high  of  14,  currently 
5)  and  researchers  transferred  to  the  center  by  the  participating  companies 
(project  high  of  over  40,  currently  30).  Additionally,  most  of  the  participating 
companies  also  conduct  FRIEND21  Project  research  at  their  own  facilities. 

The  research  activities  conducted  under  the  FRIEND21  Project  can  be  divided 
into  three  phases.  The  first  phase  extended  from  the  project  commencement  in 
1988  through  1989.  Its  primary  focus  was  the  definition  of  a  high  level  human 
interface  research  strategy.  The  complementary  concepts  of  METAPHOR,  in 
which  the  functionality  a  novice  user  wishes  to  invoke  is  represented  in  terms 
that  permit  application  of  the  user's  world  knowledge,  and  AGENCY  MODEL,  a 
computer  architecture  for  implementing  the  essential  computational  and 
interface  tasks,  emerged  as  the  unifying  research  approach  during  this  phase 
of  the  project.  Also,  the  term  METAWARE  was  coined  to  describe  the  software 
implementation  of  the  AGENCY  MODEL. 

Phase  two  of  the  FRIEND21  Project  extended  from  1989  through  1991.  During 
that  period,  a  collection  of  selected  METAWARE  and  AGENCY  MODEL  concept 
verification  research  tasks  were  pursued.  Also  during  phase  two,  several 

supporting  multimedia,  display  and  presentation  hardware  surveys  were 
conducted. 

The  concluding  phase  three  of  the  FRIEND21  Project  involved  in-depth  studies 
and  development  tasks  on  a  variety  of  information  structuring,  human 
interface  environment  development  toolsets,  METAWARE  and  AGENCY  MODEL 
topics.  The  technical  content  of  the  '94  symposium  consisted  primarily  of 
presentations  and  demonstrations  based  on  phase  three  research  projects. 

Included  in  the  products  of  the  FRIEND21  Project  are  a  total  of  76  published 
papers  (through  October  1993)  and  75  patents,  of  which  31  are  Japanese,  12  are 
U.  S.  and  33  are  European.  According  to  Mr.  Daizo  Kato,  Deputy  Director, 
FRIEND21  Research  Center,  all  intellectual  property  rights  belong  either  to  the 

Japanese  government  or  to  P.I.E.  and  are  available  to  the  public  for  a  fee. 

During  the  course  of  the  FRIEND21  project,  an  international  symposium  was 
held  at  two  year  intervals,  the  '94  International  Symposium  on  Next 
Generation  Human  Interface  is  the  last  in  the  series.  There  are  no  plans  by 
MITI/P.I.E.  to  extend  the  FRIEND21  Project  or  to  generate  any  follow  on  human 
interface  projects.  It  is  assumed  that  the  corporate  sponsors  will  continue  to 

pursue  human  interface  research  and  development  activities  on  their  own. 
However,  there  may  be  other  basic  research  projects  of  a  comparable  scale  in 
other  related  areas,  possibly  in  computer  networks  (see  section  4  below). 


2.  Sponsoring  Members: 

The  sponsoring  members  of  the  FRIEND21  Project  are  as  follows: 

Apple  Operations  and  Technologies  Japan  Branch 
IWANAMI  SHOTEN,  PUBLISHERS 
N.  T.  T.  Data  Communications  Systems  Corporation 
Oki  Electric  Industry  Co.,  Ltd. 

SHARP  CORPORATION 
The  Sumitomo  Bank,  Ltd. 

Sony  Corporation 

The  Dai-Ichi  Kangyo  Bank,  Ltd. 

DAINIPPON  PRINTING  CO.,  LTD. 

The  Mitsui  Taiyo  Kobe  Bank,  Ltd. 

TOSHIBA  CORPORATION 
TOPPAN  PRINTING  CO.,  LTD> 

IBM  Japan,  Ltd. 

NEC  Corporation 
Hitachi,  Ltd. 

Fuji  Xerox  Co.,  Ltd. 

FUJITSU  LIMITED 

MATSUSHITA  ELECTRIC  INDUSTRIAL  CO.,  LTD. 

The  Mitsubishi  Bank,  Ltd. 

MITSUBISHI  ELECTRIC  CORPORATION 

3.  The  *94  International  Symposium  on  Next  Generation  Human 
Interface 

The  '94  symposium  was  conducted  2-4  Feb.,  1994  at  the  Meguro  Gajoen  in  Tokyo. 
The  program  was  arranged  in  session  blocks  in  which  collections  of 
presentations  on  a  single  theme  were  presented.  Section  5  consists  of  abstracts 
of  all  symposium  presentations  (with  the  exception  of  the  keynote  address, 
which  is  included  in  its  entirety),  organized  by  session  block  and  including 
author  and  sponsoring  organization  information.  Included  with  some  of  the 
abstracts  are  summaries  of  the  corresponding  presentations  from  a  parallel 
summary  report  by  Dr.  Paul  Milgram,  who  is  currently  an  Invited  Researcher 
at  the  ATR  Communication  Systems  Research  Laboratories,  Kyoto,  Japan,  on 
leave  from  his  position  as  Associate  Professor  in  the  Department  of  Industrial 
Engineering,  University  of  Toronto,  Toronto,  Ontario,  Canada.  Simultaneous 
English  and  Japanese  language  translations  of  all  presentations  were 
provided,  and  virtually  all  projection  aids  were  in  English. 

The  symposium  was  attended  by  about  200  people.  With  the  exception  of  a  few 
invited  US/UK  participants  (see  the  concluding  paragraph  of  this  section), 
virtually  all  participants  were  Japanese  nationals.  The  FRIEND21  sponsoring 
organizations  were  heavily  represented. 

The  first  session  block,  conducted  Wednesday  morning,  consisted  of  keynote, 
guest  and  FRIEND21  project  background  presentations.  The  keynote  speech, 
titled  "Renewal  of  information  processing  paradigm",  was  delivered  by  Dr. 
Kochiro  Tamura,  Deputy  Director-General,  Electrotechnical  Laboratory.  As 
noted  above,  the  corresponding  paper  is  reproduced  in  its  entirety  in  section 
5.  The  Wednesday  morning  session  also  included  a  FRIEND21  project  summary 
presentation  by  Mr.  Daizo  Kato,  Deputy  Director  of  the  FRIEND21  Research 


Center  and  a  presentation  by  Professor  Masumi  Ishikawa  of  the  Kyushu 
Institute  of  Technology  on  an  attempt  (not  yet  complete)  to  formalize  a  set  of 
human  interface  design/development  guidelines.  A  post  symposium  contact 
with  P.I.E.  indicates  that  the  guidelines  will  be  published  in  June  or  July,  1994. 

The  Wednesday  afternoon  session  was  devoted  to  a  series  of  presentations, 
primarily  theoretical/academic,  on  the  "metaphor"  approach  to 
human/computer  interfaces.  Three  invited  US  participants  (see  the 
concluding  paragraph  of  this  section).  Dr.  Austin  Henderson,  Professor  Edwin 
Hutchins  and  Professor  John  Carroll  presented  papers  during  the  session. 

The  Thursday  morning  session  consisted  of  presentations  on  a  variety  of 
FRIEND21  Metaware  projects,  with  emphasis  on  metaware  (software) 

development  environments.  A  highlight  of  the  session  was  the  presentation 
by  Noburo  Asahi  of  Mitsubishi  Electric  Corporation  on  the  Metaphor 
Environment  Construction  Tools  (MECOT). 

Thursday  afternoon  was  devoted  to  Agents  and  Information  Environments. 
Professor  Pattie  Maes  from  the  MIT  Media  Lab  and  Dr.  Mik  Lamming  from  Rank 
Xerox  Research  Center,  Cambridge,  UK  delivered  presentations  during  the 
session. 

The  topic  for  the  Friday  morning  session  was  Agency  Models.  A  particularly 
interesting  project,  a  multimodal  drawing  tool  developed  at  NEC  Corporation, 

was  reviewed  by  Mayumi  Hiyoshi.  It  permits  the  user  to  employ  voice, 
keyboard  and  mouse  inputs  in  combination  to  access  a  drawing  program. 

The  concluding  session  on  Friday  afternoon  was  devoted  to  panel  discussions. 
Two  separate  panel  discussions,  one  titled  "What  we  learned  from  the  FRIEND21 
Project"  and  a  second  titled  "Toward  Future  Personalized  Information 
Environments"  were  conducted. 

In  conjunction  with  the  symposium  presentations,  an  interactive 

demonstration  project  area  was  established  and  manned  by  researchers  from 
P.I.E.  and  the  various  corporate  sponsors.  A  total  of  fifteen  demonstration 
projects  were  established  and  manned.  The  demonstrations  projects  were 
organized  into  two  broad  categories,  (1)  those  devoted  to  Metaware 
Development  activities  and  environments  and  (2)  those  devoted  to  Agency 
Model  research  tasks.  The  demonstration  projects,  all  of  which  have  associated 
symposium  presentation  abstracts  in  section  5,  are  listed  in  section  6  below. 

Although  not  directly  involved  in  the  FRIEND21  Project,  several  US/UK 
researchers  engaged  in  the  human/computer  interface  research  arena  were 
invited  to  the  '94  International  Symposium.  Following  is  a  list  of  invitees,  each 
of  whom  presented  a  paper  and  most  of  whom  served  as  panelists  in  the  two 
concluding  panel  discussion  sessions. 

Dr.  D.  Austin  Henderson,  Manager,  User  Interface  Architecture,  Corporate 
Architecture,  Corporate  Research  and  Technology,  Xerox  Palo  Alto  Research 
Center 

Professor  Edwin  Hutchins,  Department  of  Cognitive  Science,  University  of 
California,  San  Diego 


Professor  John  M.  Carroll,  Department  Head,  Computer  Science,  Virginia 
Polytechnic  Institute  and  State  University  (VPI  &  SU) 

Dr.  Michael  G.  Lamming,  Senior  Scientist,  Rank  Xerox,  Euro  Park 

Professor  Pattie  Maes,  Assistant  Professor  of  Media  Arts  and  Sciences,  Media 
Lab,  MIT 

4.  Comments 

The  MITI  sponsors  and  their  corporate  counterparts  clearly  demonstrated 

remarkable  foresight  in  selecting  this  project.  They  obviously  recognized  the 

commercial  opportunities  available  in  this  enabling  technology  arena,  and 
more  to  the  point,  over  six  years  ago  they  implemented  a  project  intended  to 
begin  the  process  of  finding  ways  to  exploit  those  opportunities.  Or  as  one 
researcher  put  it,  "it  makes  vivid  the  Japanese  recognition  that  user  interfaces 
are  gating  commercial  and  technological  opportunities  in  computing." 

However,  most  of  the  research  products  of  the  project  are  modest  at  this  point 
and  the  "metaware"  ideas  presented  here  have  not  developed  much  beyond 
those  shown  in  1991  (with  the  notable  exception  of  the  academic  work,  some  of 
which  is  very  interesting).  On  the  other  hand,  the  research  infrastructure 
residual  to  this  program  should  prove  very  valuable.  The  writer  is  not  aware  of 
any  other  comparable  focused  research  program  on  human  interfaces. 

In  addition  to  the  obvious  motives  for  pursuing  this  program,  it  appears  that 
an  ancillary  objective  is  to  answer  criticism  based  on  the  widely  held 
perception  that  Japan  does  not  do  its  "fair  share"  to  support  long-term,  basic 
research.  Observations  which  indicate  sensitivity  to  the  criticism  surface 
occasionally  in  discussions  with  project  participants,  and  indeed  it  is  almost 
stated  directly  in  the  project  summary  paper  presented  by  Mr.  Daizo  Kato, 
Deputy  Director,  FRIEND21  Research  Center  during  the  opening  session  block. 
The  ERIEND21  project  is  by  its  very  nature  a  fundamental,  long  term  basic 
research  venture  of  the  sort  which  might  discourage  corporate  investment, 
but  which  is  a  necessary  first  step.  It  is  an  effective  rebuttal  to  the  criticism.  It 
is  also  worth  noting  that  public  promotion  of  the  project  is  one  of  P.I.E.'s 
explicitly  assigned  project  responsibilities,  and  that  they  take  that 
responsibility  seriously. 

One  of  the  stated  goals  of  the  FRIEND21  project  is  that  the  human  interface 
architecture  which  emerges  from  the  project  will  develop  into  a  de  facto 
standard  for  2D' century  human  interfaces.  In  fact  a  detailed  list  of  interface 
standards  is  to  be  published  later  this  year.  However,  a  consensus  opinion 
among  consulted  symposium  attendees  is  that  this  goal  (that  the  project's 
standards  will  dominate)  is  highly  unrealistic  given  the  amount  of  commercial 
development  in  this  area,  and  that  commercial  successes  alone  will  dictate  the 
emergence  of  de  facto  standards. 

Another  difficulty  may  be  that  the  decision  to  focus  on  architecture,  standards, 
and  guidelines  rather  than  to  implement  robust  systems  has  limited  the 
possible  impact.  The  agent  part  of  the  project  is  very  close  to  what  many 
groups  in  the  West  are  doing  but  real  systems  work  would  have  helped  to 
clarify  thinking  and  to  make  compelling  what  has  been  done.  However,  a 
reasonable  amount  of  interesting  pre-competitive  research  ideas  have  been 
produced  (Hirose's  theory  is  an  advance  on  previously  published  papers)  and 


there  are  many  concrete  directions  to  go  from  that  work.  Perhaps  some 
proprietary  efforts  have  already  been  spawned  by  FRIEND21  but  not  shown 

A  somewhat  puzzling  observation  is  that  there  appears  to  be  little  or  no 
connection  or  cooperation  between  FR1END21  and  TRON  (The  Real  Time 
Operating  System  Nucleus),  which  also  includes  a  significant  amount  of 
human  interface  related  research  activity.  In  fact,  the  demonstrations  of 
human  interface  at  the  TRON  Symposium  seemed  more  ingenious  than  the 
mostly  prosaic  ones  shown  here. 

Einally,  an  obvious  question  is  what  P.I.E.  will  do  following  FRIEND2rs  March, 
1994  conclusion.  The  FRIEND21  project  is  by  far  the  largest  activity  on  its  plate. 
As  noted  in  section  1  above,  there  are  no  plans  to  extend  FRIEND21  or 
implement  a  follow  on  to  it.  However,  there  are  indications  that  another 
project,  comparable  in  scope  and  purpose,  may  be  in  the  works  and  that  it  will 
probably  focus  on  computer  networking. 

5.  Papers  Presented  at  the  *94  International  Symposium: 

Following  is  the  symposium  keynote  address  by  Dr.  Kochiro  Tamura  and 
abstracts  of  all  other  papers  presented  at  the  symposium.  The  paper/abstracts 
are  organized  in  the  session  block  order  discussed  in  section  3  above.  Also 
included  with  some  of  the  abstracts  are  summaries  of  the  corresponding 
presentations  from  a  parallel  summary  report  by  Dr.  Paul  Milgram,  who  is 
currently  an  Invited  Researcher  at  the  ATR  Communication  Systems  Research 
Laboratories,  Kyoto,  Japan,  on  leave  from  his  position  as  Associate  Professor  in 
the  Department  of  Industrial  Engineering,  University  of  Toronto,  Toronto, 
Ontario,  Canada  (e-mail  milgram@atr-sw.atr.co.Jp).  Each  of  Dr.  Milgram's 
summaries  appears  in  brackets  immediately  following  the  corresponding 
abstract. 

Dr.  Tamura's  address  and  telephone/FAX  numbers  are  as  follows. 

1-1-4  Umezono 
Tsukuba 
Japan,  305 


Tel:  +81-298-54-5003 
Fax:  +81-298-54-5070 


Keynote,  Guest  and  Project  Background  Presentations 


Renewal  of  information  processing  paradigm 


Koichiro  Tamura 
ktamura@etl.go.jp 

Electrotechnical  Laboratory 


1.  Recent  paradigm  shift  in  information  industry 

We  are  in  the  age  of  rapid  change  of  computer  technology.  Although  the 

computer  history  of  50  years  has  had  incessant  revolution,  the  recent  change 
seems  particularly  drastic.  To  sum  up  the  trend  in  the  last  decade,  industrial 
weight  shift  has  occurred,  from  hardware  to  software  and  in  software  from 
order-made  to  packaged  one.  The  so  called  "Information  society"  emerges  in 
this  appearance.  It  may  be  predicted  that  the  information  companies  that 
produce  packaged  software  with  good  quality  will  prosper  but  those  who  rely 
only  on  hardware  or  ordered  software  will  decline. 

Down-sizing  phenomena  is  of  course  in  this  trend.  Most  users  are  getting 
away  from  main  frames  and  prefer  personal  computers  and  workstations. 
Why?  Because  the  hardware  is  getting  cheaper.  Why?  Because  they  can  sell 
many.  Why?  Because  the  software  that  has  nice  functions  with  ease  of  use 
was  developed.  If  it  is  still  difficult  to  use  for  many  people,  they  would  surely 
never  use  computers  even  if  they  are  cheap.  The  market  would  be  limited  and 
small.  Then,  who  would  owe  the  enormous  cost  of  research  and  development 
that  is  essential  to  reduce  the  cost?  Thus,  some  good  packaged  software  with 
ease  of  use  enlarged  the  market,  that  prompted  the  investment  to  develop 
hardware  technology,  that  in  turn  increased  the  number  of  users 
tremendously. 

Those  are  phenomenal  shifts  in  the  recent  market  and  technology.  I  foresee, 
however,  that  more  fundamental  paradigmatic  changes  of  information 
technology  shall  be  necessary  and  actually  about  to  happen  to  meet  the  users’ 
satisfaction  in  the  future. 

2.  From  "  ease  to  use  "  to  "  ease  to  make  " 

One  can  clearly  divide  the  information  technology  into  hardware  and 
software.  As  my  definition,  software  unit  is  an  integration  of  programs  and 
data  those  are  supposed  to  be  used  by  unspecified  users.  The  remarkable 
characteristics  of  the  software  are  (1)  ease  of  copy,  and  (2)  loadability  of 
multiple  units  for  one  hardware.  The  success  of  Nintendo's  game  machine 
business  is  due  to  the  shift  to  software  after  giving  up  to  produce  each 
hardware  for  each  game.  Computer  hardware  is  general  purpose  with  no 
function,  and  the  software  gives  the  function  that  is  easily  copied  with  few 


cost.  Computers  are  definitely  different  from  TV  or  automobiles  in  this  point. 
Users  buy  computers  in  order  to  use  some  software  that  gives  function. 

It  seems  obvious  that  the  relative  importance  of  software  to  hardware  is 
increasing  in  terms  of  market  sales,  production  cost,  or  the  number  of 
employees  engaged  in'.  Unfortunately,  we  do  not  have  exact  statistics  that 
shows  this  phenomenon,  but  our  perception  tells  us  that  the  weight  of 
software  has  already  exceeded  over  the  one  of  hardware  in  the  industry. 

This  phenomenon  appears  not  to  be  limited  only  in  information  industry. 

Most  kind  of  industries  and  many  kind  of  social  activities  are  strongly  affected 
by  software-oriented  systems.  Information  society  means  to  see  this 
phenomenon  in  all  branches  of  our  social  activities.  It  is  not  a  rare  ease  for 
one  automobile  to  have  more  than  100  microprocessors.  The  total  software 
for  those  microprocessors  must  be  huge.  It  is  said  that  a  recent  high-tech  jet 
fighter  loads  software  with  over  I  million  steps.  The  designers  of 
automobiles  or  plains  definitely  need  supercomputers  with  huge  CAD 
software.  To  sum  up,  information  society  means  software-shifted  society. 

The  progress  of  software  technology  has  been  changing  the  way  to  use 
software.  Computers  used  to  be  machines  for  users  to  program  by  themselves. 
Most  users  nowadays  make  use  of  software  to  have  computers  work  as  they 
want.  In  addition,  less  users  want  order-made  software  but  packaged  ones.  In 

Japan,  this  change  is  occurring  years  behind  other  countries,  that  causes  not 
only  the  lag  of  the  software  technology  but  also  inefficient  structure  of  the 
whole  system  of  the  country.  It  is  obvious  that  mass  produced  (production  of 
software  means  duplication  with  little  cost),  inexpensive  software  with  high 
quality  is  much  more  efficient  than  order-made  one.  Unfortunately,  such 
good  packages  are  still  few  in  Japan.  The  discussion  on  this  matter  is  beyond 
the  scope  of  this  paper. 

Although  most  users  of  more  than  30  million  personal  computers  in  the 
world  make  use  of  packaged  software,  they  are  not  completely  satisfied  with 
those.  No  paekage  can  meet  completely  what  users  want.  That  is  why 
customization  is  necessary,  and  so  called  end-user  computing  are  strongly 
demanded.  Of  course,  most  package  software  prepares  some  method  for  users  to 
customize,  but  most  users  feel  those  are  too  difficult  to  exploit.  Urged  here  is 
the  shift  from  "ease  to  use"  to  "ease  to  make".  In  this  sense,  the  way  to  use 
computers  seems  to  change  back  to  the  old  age  when  users  programmed 
computers  to  use  by  themselves  as  they  want. 

3.  Network  age  and  software 

Computer  network  has  brought  many  to  many  communication  whereas  the 
telephone  is  one  to  one  and  the  broadcast  is  few  to  many.  People  can  enjoy  a 
completely  new  kind  of  world  by  "surfing"'  on  the  networks  to  communicate 
with  other  people  and  databases,  small  or  large,  and  machines  running 
interesting  programs,  however  far  away  they  are.  Networking  is  thus  the 

^For  example.  Dr.  Sekimoto  ,  President  of  NEC,  said,  "I  think  software  will  be  much 
important.  I  would  exaggerate  that  one  who  conquers  the  software  conquers  the  world. 
Anyway,  we  foresee  the  age  of  the  software  with  much  importance."  (JAIDA  Journal  35,  7 
p.2  1993) 


important  side  of  information  society,  and  its  point  is  again  its  software;  The 
software  to  back  up  and  be  delivered  through  the  network  is  vital  to  attract 
users. 

The  software  supplied  through  network  includes  not  only  computer  software 
and  date  in  databases  but  also  CD  music,  video  and  multimedia-type.  In 
addition,  services  such  as  barber  shops  that  have  been  confined  only  locally 
thus  far  may  be  performed  remotely  through  high  speed  network  and  remote 
operation  technique.  Even  surgery  operation  can  be  done  through  long  haul 
network.  Many  sort  of  services  can  be  carried  out  on  networks  and  distributed 
around  the  world.  This  is  another  big  change  we  will  see  in  the  near  future. 

4.  Computers  in  the  near  future 

Recent  software  consume  immense  memory.  Even  simple  Jobs  in  office 
automation  often  need  more  than  8  million  byte  RAM.  If  people  want 
multimedia  or  AI  solution,  they  may  need  more  than  100  million  bytes. 

In  the  near  future,  a  single  user  may  use  more  than  one  computers  at  a  time. 
Even  now,  we  can  see  those  who  use  2  to  3  workstations  and  personal 
computers  on  his/her  desk.  If  it  is  the  case,  we  want  some  conditions  to  such 
computers  as  the  followings. 

(1)  cheap, 

(2)  light, 

(3)  wide, 
and 

(4)  large  capacity  of  communication  with  the  user. 

Any  computer  that  meets  those  conditions  may  look  like  a  piece  of  paper, 
which  is  light,  thin,  wide  with  high  resolution  to  input  and  output.  In  fact, 
FRIEND  21  Project  supposed  this  kind  of  computers  (we  named  "papyrus 
computer')  and  has  pursued  human  interface  system  for  those  machines. 

Users  would  use  some  papyrus  computers  on  his/her  real  desk  like 
windows  in  desk  top  metaphor  of  the  today's  computers.  Whiteboard  size 
papyrus  computers  may  be  necessary.  This  prediction  and  expectation  was 
held  about  6  years  age,  but  we  think  that  this  foresight  is  still  effective  and 
may  come  true  in  the  first  decade  of  the  2D*  century^. 

Needles  to  say,  those  machines  may  be  connected  with  each  other,  hopefully 
by  wireless.  The  human  interface  for  them  contribute  to  hide  away  the 
mechanism  of  computation  as  if  users  feel  to  use  something  but  computers. 

GUI  is  one  of  the  technique  to  hide  away  computers  from  users  view.  A  large 
number  of  micro-computers  are  already  exploited  actually  behind  the  scene, 
embedded  in  many  kinds  of  industrial  productions. 

Users  will  never  want  a  large  capacity  of  memory  accidentally  evaporated. 
They  should  be  non  -volatile,  that  urges  the  necessity  of  the  research  and 
development  for  new  type  of  RAM.  Battery  problem  also  needs  some  solution. 
They  should  have  much  longer  life  and  be  much  thinner  and  lighter. 

^Recently,  the  Ubiquitous  Computing  Project  by  XEROX  PARC  is  pursuing  the 
similar  concept. 


5.  Computers  in  the  far  future 

In  the  far  future,  how  large  memory  do  people  want  to  use?  I  predict  they  will 
want  it  infinitely,  if  they  can  afford.  In  fact,  CAD  users  nowadays  use 
workstations  with  a  few  giga  bytes  RAM.  It  is  possible  for  clerks  in  offices 
would  want  the  same.  Software  itself  grows  larger  exponentially  with  much 
more  complexity  and  functionality,  as  well  as  data  such  as  multimedia. 

Software  people  should  be  afraid  if  it  is  possible  forever  to  be  supplied  such 
enormous  memory  to  cover  the  incessantly  increasing  size  of  programs  and 
data.  How  about  to  accommodate  Al  programs?  The  essence  of  A1  computation  is 
search,  which  needs  certainly  very  large  size  of  memory  space,  and 
parallelism  that  of  course  needs  much  more  hardware. 

In  nowadays,  people  do  not  believe  in  the  effectiveness  of  Al  technology.  The 
Fifth  Generation  Project  seems  to  show,  however,  that  it  is  not  software's 
shortfall  but  hardware's  to  back  up  the  advanced  software,  since  the  project 
had  developed  languages  and  systems  on  the  assumption  that  parallelism  be 
available,  and  eventually  resolved  many  difficult  Al  problems. 

Some  hardware  people  predict  the  limit  of  memory  chip  technology  as  several 
giga  bits  per  chip.  Right  or  wrong,  there  surely  is  some  limit  for  any 
technology  and  the  size  of  software  will  not  stop  increasing,  although 
improvement  of  algorithm  and  techniques  still  goes  on.  We  would  see  in 
decades  a  kind  of  catastrophe  if  we  cannot  change  drastically  the  essential 
paradigm  of  computation. 

6.  Essential  renewal  of  computational  paradigm 

Not  only  the  limitation  of  hardware  development  urges  the  renewal  of  the 
computational  paradigm.  Even  if  Al  technology  has  developed  to  some 
advanced  level  by  the  tong  efforts  such  as  Fifth  Generation  Project,  human 
intelligence  is  still  mysterious  from  the  standpoint  of  the  information 
technology  to  date.  For  example,  no  computer  can  learn  how  to  speak  any 
natural  language  like  a  human  infant  easily  does.  The  computation 
technology  would  be  frozen  ever  as  the  automobiles  with  gasoline  engine,  if 
we  cannot  find  completely  different  paradigm  to  mechanize  intelligence. 

The  sentiment  to  demand  this  kind  of  paradigm  shift  has  spotlighted  "new" 
exotic  computation  concepts  such  as  neural  network,  fuzzy  theory,  genetic 
algorithm,  chaos  theory  and  A-life.  Those  are,  however,  not  new,  and  actually 
decades  older.  I  may  predict  that  none  of  those  will  take  over  the  today's 
computation  paradigm,  although  some  can  be  supplemental  to  it.  We  need 
completely  new  representation  of  information  and  way  of  computation. 

In  terms  of  device  technology,  it  is  doubtful  that  it  is  effective  forever  to 
continue  to  pursue  smaller  unit  cells.  The  another  way  to  find  a  right  model  in 
biological  systems  is  also  doubtful,  because  it  is  too  complex  to  understand  the 
mechanism  of  biosystems  from  our  knowledge  to  date.  Research  on  the  neuron 
and  brain  mechanism  appears  to  me  to  presuppose  that  computation  models 
working  In  the  biosystem  are  similar  to  the  today's  computer  mechanism.  I 
doubt  if  they  can  find  out  any  new  paradigm  from  this  kind  of  approach.  New 
paradigm  for  intelligence  should  be  find  out  by  serious  thought  upon  human 
intelligence  as  philosophical  or  religious  thinking  since  thousands  years  ago. 


Computer  people  may  have  a  good  chance  to  generate  a  brand  new  philosophy 
through  their  pursuit  of  mechanizing  intelligence. 


Summary  of  the  FRIEND21  Project 
Daizo  Kato 

FRIEND21  Research  Center 

Institute  for  Personalized  Information  Environment 

The  FRIEND21  Project  began  its  research  activities  in  1988  with  the  goal  of 
creating  a  human  interface  for  the  21^‘ century  that  allows  for  the  pleasant, 
convenient,  and  efficient  use  of  computers.  For  five  years,  research  into 
human  interface  has  been  carried  out  in  various  ways  with  the  help  of 
participating  companies,  such  as  computer  manufacturers,  home  electronics 
manufacturers,  publishing  companies,  and  printing  companies.  The  FRIEND21 
Project  has  proposed  new  design  principles  for  the  2lsCcentury  human 
interface  using  contextual  metaphors  called  "metaware",  and  new  software 
architecture  called  the  "agency  model"  for  the  implementation  of  metaware. 
We  are  now  researching  an  "integration  model",  that  systematically  covers 
metaware,  the  agency  model  ,  and  design  environments  for  these  frameworks 
and  architectures. 

At  the  end  of  March  1994,  the  FRIEND21  Project  will  conclude  its  six  year 
research  project.  The  results  of  this  project  are  now  and  will  continue  to  be 
open  to  the  public. 


Toward  Guidelines  for  Friendly  Interfaces 
Masumi  Ishikawa 
Kyushu  Institute  of  Technology 

At  the  outset  I  will  present  brief  history  of  standards  in  the  field  of 
information  technology.  In  a  human  interface,  which  is  one  of  the  subareas 
of  information  technology,  the  immaturity  and  the  rapid  progress  of  its 
technology  makes  the  standardization  difficult.  Hence,  the  need  arises  for 
guidelines  and  recommendations  for  the  design  of  human  interfaces.  In  the 
FRIEND21  project  a  novel  human  interface  architecture  comprising  metaware 
and  an  agency  model  is  proposed.  It  has  pursued  guidelines  and 
recommendations  based  on  this  architectures  as  one  of  its  major  results.  I  will 
outline  the  efforts  to  date  toward  the  guidelines  and  briefly  summarize  the 
project  from  the  point  of  view  of  design  guidelines  and  recommendations. 


An  Agenda  for  HCI 

Austin  Henderson 
Manager,  Use  Architecture 
Xerox  Corporation 

This  paper  presents  my  personal  assessment  of  where  the  field  of  Human- 
Computer  Interaction  (HCI)  is  today,  and  a  few  key  challenges  it  faces.  This  is 
presented  in  two  parts:  First,  I  discuss  the  current  common  framing  of  HCI, 
argue  that  it  is  centered  on  bringing  a  person  to  a  computer,  and  propose  that 


it  be  re-centered  on  bringing  computation  to  the  everyday  activity  of  people. 
Second,  based  on  this  change  in  perspective,  I  discuss  a  few  aspects  the 
activity  that  people  do,  particularly  those  employing  technology,  and  for  each 
show  how  we  can  improve  the  quality  of  HCI  by  better  fitting  in  with  the 
richer  interactions  of  people's  activity. 

[Austin  Henderson,  Manager  of  Use  Architecture  at  Xerox  Corp.,  presented  his 
personal  view  of  the  current  emphasis  in  the  field  of  Human-Computer 
Interaction  (HCI)  towards  one  person  sitting  in  front  of  one  computer,  and 
how  computation  is  destined  to  become  more  'ubiquitous'  as  it  permeates 
activities  of  people  in  everyday  life,  both  inside  and  outside  the  office.] 


Metaphor  Presentations 


Metaphors  for  Interface  Design 

Edwin  Hutchins 

University  of  California,  San  Diego 

Computers  are  the  most  plastic  medium  ever  invented  for  the  representation 
and  propagation  of  information.  In  fact,  they  are  so  adaptable  and  can 

manifest  such  a  wide  range  of  behaviors,  that  little  but  the  hardware  itself 
may  be  easily  identifiable  as  an  enduring  property  of  the  device.  Computers 
can  mimic  the  behaviors  of  other  information  media  and  can  manifest 
behaviors  that  are  simply  not  possible  in  any  other  medium.  We  might  speak 
literally  about  the  nature  of  the  computer's  behavior  (to  the  extent  we  can 
speak  literally  about  anything)  at  a  very  low  level,  describing  the  changes  in 
the  states  of  silicon  gates  and  so  on,  but  even  there  we  frequently  resort  to 
metaphors.  As  the  levels  of  complexity  are  layered  one  atop  the  other  to 
produce  the  high-level  behaviors  that  are  the  actions  we  recognize  while 
interacting  with  the  computer,  the  possibility  of  talking  or  thinking  literally 
about  the  computer's  behavior  vanishes.  We  deal  with  this  complexity  and  this 
plasticity  by  speaking  metaphorically  about  the  behavior  of  the  computer. 

The  metaphors  we  use  both  intentionally  and  unintentionally,  contribute 
structure  in  terms  of  which  we  organize  our  understandings  of  what  is  going 
on  (Lakoff  &  Johnson,  1980). 

[Edwin  Hutchins  of  the  University  of  California  at  San  Diego  gave  a  talk  on  the 
underrated  importance  of  metaphors  for  affecting  the  feel  of  interfaces  and 
influencing  the  development  of  technology.  His  presentation  gave  a 
classification  of  different  types  of  metaphors,  under  the  headings  of  activity 
metaphors,  mode  of  interaction  metaphors  and  task  domain  metaphors.  He 
subsequently  concentrated  on  conversation  metaphors,  model-world 
metaphors  and  collaborative  manipulation  metaphors,  all  with  respect  to  mode 
of  interaction.  It  was  especially  encouraging  to  see  that  Professor  Hutchins 
recognizes  the  fact  that  computer  interfaces  must  go  beyond  merely  enabling 
communication  between  the  user  and  software  packages,  but  must  also  take 
into  account  real-world  tasks,  as  in  process  and  vehicular  control 
applications.] 


The  role  of  reflection  in  understanding  systems 

Naomi  Miyake 

School  of  Computer  and  Cognitive  Sciences 

Chukyo  University 

I  have  been  interested  in  how  reflection  works  in  understanding  and 
thinking.  The  problem  of  reflection  has  been  identified  and  talked  about  in 
cognitive  science.  As  one  example,  Don  Norman  maintains  in  one  of  his  recent 
books  that  there  are  two  modes  of  cognition,  experiential  and  reflective 

(Norman,  1993). 

Besides  his  observations,  experimental  studies  have  shown  that  laborious, 
reflective  activities  have  their  own  merits  (Miyake,  1990;  1991).  For  any 
interface,  when  one  uses  it,  there  occur  reflective  phases  during  the  process. 

Identifying  and  analyzing  such  reflection  might  be  useful  for  understanding 
better  principles  for  interface  design. 

While  you  are  at  work,  you  can  reflect  upon  what  you  are  doing  at  the  moment, 
on  your  own  actions  and  on  the  reactions  and  feedback  you  get  from  the 
environment.  As  another  type  of  reflection,  you  can  also  reflect  upon  the 

mentally  recreated  version  of  such  a  process,  perhaps  not  at  the  moment,  even 

without  the  same  physical  set  up  in  the  environment. 

If  we  take  an  example  of  people  trying  to  understand  how  to  use  an  application 
software  on  a  computer,  the  former  is  a  part  of  what  they  think  while  they  are 
trying  its  commands,  getting  reactions,  feedback  and  everything.  The  latter  is 
what  they  remember  and  think  over  later,  perhaps  away  from  the  computer, 
by  mentally  looking  back  on  their  experience.  I  have  been  mainly  interested 
in  this  latter  type  of  reflection,  because  it  is  a  mental  process  on  a  mental 
process,  which  has  a  potential  of  creating  and  modifying  some  possibly  new 
perspective  not  bound  by  the  physical  constraints  of  reality.  This  could  give 
us  some  freedom  to,  for  example,  stretch  the  limits  of  an  action  to  see  new 
possibilities.  For  the  reflection  on  mental  models  to  work,  the  mental  models 
themselves  have  to  be  the  decent  representation  of  what  could  happen  in  the 
real,  physical  world. 

I  have  been  interested  in  finding  out  how  the  latter  type  of  reflection  leads  to 
new  perspectives,  different  ways  to  see  the  problem  at  hand,  to  new  creation. 
Such  findings  should  be  useful  for  designing  technological  support  for 
intellectual  activities.  In  this  paper  I  use  the  term  "Reflection"  to  refer  to  this 
type  of  off-the-spot  reflection.  The  main  questions  are,  how  the  reflection 
promotes  understanding,  how  much  direct,  interactive  experience  we  need  to 
build  workable  mental  models  for  effective  reflection,  what  extent  of  physical 
detachment  is  needed  for  establishing  the  reflection  and  the  like.  I  try  to 
answer  some  these  questions,  taking  support  from  quasi-experimental 
research. 


Binding  Metaphors  to  Scenarios  of  Use 
John  M.  Carroll 
Professor  and  Head 
Department  of  Computer  Science 
Virginia  Polytechnic  Institute  and  State  University 
Blaeksburg,  Virginia  24061-0106 


Metaphor  has  proven  to  be  one  of  the  richest  and  most  robust  ideas  in  the 
design  of  computer  applications  and  user  interfaces.  The  basic  idea  is  very 
simple;  present  functionality  in  such  a  way  that  the  user  can  access  and  apply 
specific  prior  knowledge  while  learning  and  using  a  novel  tool.  But  the 
practical  and  theoretical  ramifications  of  this  idea  both  in  the  brief  history  of 
human-computer  interaction  and  in  its  current  prospects  are  quite 
considerable.  In  this  talk,  I  first  summarize  my  view  of  the  relevant  history.  I 
then  develop  the  notion  that  metaphors  should  be  conceived  of  as  bound  to 
contexts  of  use:  The  recognition  and  interpretation  of  metaphors  typically 
depends  upon  the  establishment  of  a  meaningful  task  context.  I  think  we  need 
to  focus  our  consideration  of  metaphors  on  the  scenarios  of  use  from  which 
they  arise.  I  suggest  that  this  recognition  of  metaphors  as  bound  to  scenarios 
of  use  converges  with  recent  developments  in  scenario-based  specification 
and  object-oriented  design. 

[John  Carroll  of  Virginia  Polytechnic  Institute  also  argued  the  very  important 
point  that  research  on  metaphors  must  take  into  account  the  task  context,  or  in 
other  words  that  scenarios  of  use  must  be  developed  as  an  integral  part  of 
interface  design.  Thus  how  the  metaphor  relates  to  the  context  of  the  task 
being  performed  is  possibly  much  more  important  than  the  actual  (physical) 
metaphor  itself,  which  has  most  likely  been  developed  outside  of  the  task 
context.] 


Interface  Metaphors  and  their  Roles  in  the  learning  of  Computer  systems 
Takashi  Kosumi 

Institute  of  Socio-Economic  Planning 
University  of  Tsukuba 
Hiroshi  Nunokawa  and  Kiwamu  Sato 
Research  Institute  of  Electrical  Communication 
Tohoku  University 

This  paper  addresses  the  roles  of  interface  metaphors  in  the  learning  of 
computer  systems.  There  are  several  earlier  articles  on  interface  metaphors 
(e.g.,  Carroll,  Mack,  &  Kellogg,  1988;  Henderson  &  Card,  1986;  Hirose,  1991; 
Kusumi  &  Matsubara,  1993;  Mander,  Salmon,  &  Wong,  1992;  Nonogaki,  1990). 
However,  only  a  few  studies  have  been  presented  on  typology  of  interface 
metaphors.  This  study  ;had  three  clear  aims.  First,  we  classify  interface 

metaphors  based  on  their  cognitive  function.  Second,  we  proposed  multiple 

interface  metaphor;  DoReMi  (Sato,  Nunokawa,  &  Noguchi,  1993),  a  graphical 
user  interface  for  computer  network  systems  based  on  the  city  metaphor. 

Third,  we  evaluate  the  interface  metaphor  by  psychological  experiment. 

[Takashi  Kusumi  of  the  University  of  Tsukuba  dealt  with  the  importance  of 
metaphor,  analogy  and  personification  in  connecting  domains,  and  of 
metonymy  and  synecdoche  in  connecting  components  within  domains.  Such 
concepts  are  of  primary  importance  in  facilitating  the  learning  of  new 
systems  by  novices.  In  addressing  this  need,  Kusumi-san's  group  has  developed 
the  DoReMi  system  which  is  based  on  the  metaphor  of  a  city.  Illustrations  and 
evaluation  results  were  given  of  tasks  such  as  address  look-ups  via  the 
metaphorical  city  office  and  Email  posting  through  the  metaphorical  city  post 
office.] 


Metaware  Project  Presentations 


An  Environment  for  Developing  Metaphor  Worlds 
-  Toward  a  User-friendly  Virtual  work  Space  based  on  Metaware 

Nobuo  Asahi 

Mitsubishi  Electric  Corporation 

One  of  the  most  important  issues  on  Human  Interfaces  is  giving  users  suitable 
system  images.  Especially  for  non-experl  users,  the  system  image  should  be 
easy  to  understand  without  special  knowledge  of  computers.  Interface 
metaphors,  which  were  primarily  applied  to  the  Xerox  Star  system  (1),  have 
been  playing  significant  roles  in  giving  easy  system  images  of  eomplex 
computer  applications  by  adopting  well-known  entitles  as  user  interface 
objects. 

In  the  current  graphical  user  interface  toolkits  (GUI  toolkits,  for  sort),  the 
notion  of  interface  metaphors  is  partially  used  by  preparing  class  objects 
which  represent  well-known  entitles,  such  as  buttons,  sliders,  menus  and  so 
on.  Each  of  interface  metaphors  is  easy  to  understand  how  to  operate, 
however,  the  composed  user  interface  often  fails  to  show  a  system  image  of  an 
application.  This  is  because  the  GUI  toolkits  are  designed  to  be  general  and 
simple  so  as  to  be  applied  various  application  interfaces,  and  because  most  of 
them  are  supplied  as  the  sets  of  given  class  objects  which  make  it  difficult  to 
create  new  class  objects  necessary  to  construct  a  suitable  system  image. 

Further  more,  as  the  GUIs  are  constructed  by  arranging  objects  two 
dimensionally,  the  screen  size  of  a  computer  display  would  be  a  constraint  to 
its  design  potential. 

[Asahi  (Mitsubishi  Electric)  reported  on  the  MECOT  (Metaphor  Environment 
Construction  Tools)  system.  Current  graphical  user  interfaees  (GUI's)  usually 
lack  the  system  image  of  an  application,  and  since  many  systems  are  three 
dimensional,  a  tool  for  creating  3D  metaphor  environments  is  desirable.  The 
MECOT  system  allows  for  rapid  construction  of  various  interfaces.  Its 
architecture  consists  of  Metaphor  Editor,  Environment  Manager  and 
Environment  Builder.  The  Editor  is  used  for  creating  objects.  The  Environment 

Builder  is  used  for  putting  parts  together  to  construct  an  environment,  such  as 

an  office,  with  desks,  drawers,  filing  cabinets,  etc.  in  3D.  It  was  claimed  that 
MECOT  is  suitable  for  rapid  prototyping  (95%  fewer  steps  in  comparison  with  C 
programming)  and  that  the  3D  animation  capability  encourages  designers. 

Two  tests  were  conducted.  The  first  experiment  (2  subjects)  indicated  that  the 
3D  environment  was  easier  to  understand  than  the  2D  environment,  but  it  was 

difficult  to  know  the  distance  between  objects  in  3D  and  difficult  to  manipulate 
objects  in  3D.  The  second  experiment  (5  subjects)  showed  that  local  co¬ 
ordinates  based  on  moving  objects  are  best  for  object  movement.] 


Class  Based  Dynamic  User  Interface  Environment 
Masaya  Ueda 
Sharp  Co. 

We  introduce  an  approach  to  build  dynamic  user  interface  environment  on 
class  based  object  oriented  system.  The  world  "dynamic"  means  that  instances 
in  the  object  oriented  system  can  change  their  behavior  after  they  are 


created.  Such  a  dynamic  changeability  is  very  advantageous  feature  to  built 
flexible  user  interface  environment. 

Static  (i.e.,  conventional)  class  based  object  oriented  systems  have  the 
constraints  on  changeability  and  adaptability  of  instanee  behavior  that  result 
from  binding  instances  irrevocably  to  a  class.  In  these  systems,  instances  can 
not  change  their  behavior  after  they  are  created. 

In  the  real  world,  we  often  bind  some  pieces  of  paper  into  a  file.  Static  class 
based  systems  can  not  simulate  the  equivalent  operation  even  though  it  is  a 
very  basic  action  in  our  daily  life. 

In  class  based  environments,  each  paper  and  file  has  a  corresponding  class. 
Suppose  that  you  created  some  new  pieces  of  paper  from  their  classes  and 
wrote  something  on  them;  but  you  can  not  bind  them  into  a  file.  They  must 
change  their  behavior  to  be  bound  into  a  file.  For  example,  they  must  have 
new  functions  such  as  "next  page,"  "previous  page,"  etc.;  but  they  can’t.  Of 
course,  you  can  make  a  new  file  instance  from  its  class,  but  it  is  a  file  of  blank 
papers.  Similarly  you  can  not  take  out  a  leaf  from  a  file. 

To  bind  existing  pieces  of  paper  into  a  file,  they  must  change  their  behaviors 
without  losing  each  identity  (i.e.,  text,  picture,  color,  cursor  position,  font, 

etc.).  But  in  static  class  based  systems,  instances  can  not  change  their 
behavior  after  they  are  created. 

Instance  based  programming  is  a  solution  to  get  more  flexibility  and 
adaptability,  but  we  took  another  approach.  We  use  class  based  system,  and 
change  the  class  of  an  existing  instance  dynamically  to  change  its  behavior. 

This  way  can  comply  with  sometimes  contradictory  two  requests,  flexibility 
and  robustness.  We  use  Common  Lisp  Object  System  (CLOS)  on  Macintosh 
Common  Lisp  (MCL). 

[Ueda  (Sharp)  reported  on  their  work  on  class  based  dynamic  user  interface 

environments.  The  keyword  here  is  "dynamic",  which  refers  to  the  fact  that 

the  end  user  can  change  the  behaviour  of  an  interface  after  it  is  generated,  as 
opposed  to  conventional  static  object  oriented  systems,  in  which  objects  are 
bound  to  a  certain  class.  Since  in  the  real  world  we  often  desire  to  change  the 
attributes  of  objects  (for  example,  by  taking  individual  pieces  of  paper  and 
combining  them  into  a  file)  dynamic  classes  should  have  many  advantages  (if 
they  don't  become  too  confusing  when  too  much  flexibility  is  provided.)] 


Visible  Software  Parts  in  Human  Interface  Design 
Noriyuki  Iwane, 

Toshihiko  Shirokaze, 
and  Tetsuo  Kinoshita 
Oki  Electric  Industry  Co.,  Ltd. 

In  a  future  personalized  information  processing  environment,  users' 
requirements  for  human  interface  (HI)  will  be  more  diversified  than  now. 
According  to  this,  an  important  problem  is  how  efficiently  to  design  a  HI 
which  respective  users  will  be  satisfied.  We  have  studied  this  problem  from  a 
viewpoint  of  presentation  of  Hi  and  proposed  an  effeetive  design  method  of  HI> 
Under  the  proposed  method,  the  software  parts  whieh  provide  visible  elements 
of  a  HI  are  called  visible  software  parts,  and  then  a  framework  for  developing 
the  visible  software  parts  has  been  proposed  in  conjunction  with  FRIEND21 
architecture.  The  visible  software  parts  are  defined  together  with  various 
knowledge  concerning  the  use/reuse  of  these  parts.  Using  these  knowledge,  it 


will  be  able  to  select  and  utilize  suitable  visible  software  parts  in  a  design 
process  of  a  HI  based  on  users'  requirements. 

Various  parts  knowledge  of  visible  software  parts  are  mapped  and  represented 
in  formal  descriptions.  And  then,  these  descriptions  are  also  reformed  into 
another  formal  descriptions  in  order  to  retrieve  visible  software  parts 
efficiently.  As  a  result  of  above  reformation,  parts  knowledge  is  represented 
by  two  kinds  of  knowledge  modules  called  logical  parts  and  physical  parts.  A 
logical  parts  represents  knowledge  of  the  use  of  visible  software  parts  with 
respect  to  users'  task.  On  the  other  hand,  a  physical  parts  represents 

knowledge  of  the  usage  of  visible  software  parts  in  a  HI  software  system  to  be 

designed.  According  to  these  logical/physical  parts,  the  design  support 

mechanisms  of  systematic  acquisition,  representation,  and  utilization  of 
knowledge  to  use/reuse  various  visible  software  parts  for  HI  design,  can  be 

realized. 

In  section  2,  a  framework  for  representing  various  knowledge  regarding 
visible  software  parts  in  proposed  based  on  a  discussion  of  the  visible  software 
parts  and  a  design  task  of  a  human  interface  system.  In  section  3,  a  design 
method  of  a  human  interface  based  on  visible  software  parts  is  demonstrated 
and  the  proposed  method  is  also  evaluated  in  section  4. 

[Iwane  et  al  (Oki  Electric  Industry)  emphasized  the  large  expense  associated 
with  designing  and  implementing  user  demands  into  individual  interfaces, 
especially  as  interfaces  become  more  diversified,  and  how  it  is  logically  more 

economical  to  reuse  portions  of  software  that  have  already  been  designed.  In 
addressing  this  problem,  they  classify  software  parts  which  provide  visible 
elements  of  a  human  interface  as  "visible  software  parts",  and  in  the  talk  they 
discussed  the  framework  for  representing  these  visible  software  parts  and 
incorporating  them  into  new  interfaces.  ] 


Sound  Metaphors  in  3D  Space 

Michio  Miwa,  Hiroyuki  Hikita,  Miwa  Fukino 
Masahiko  Koizumi 

Tokyo  Information  Systems  Research  Laboratory 
Matsushita  Electric  Industrial  Co.,  Ltd. 

Sound  plays  an  important  role  in  the  operation  and  use  of  computers.  To  look 
further  into  the  role  played  by  sound,  sound  must  be  arranged  in  a  practical, 
i.e.,  3-dimensional  environment  and  studied  from  different  viewpoints.  And, 
to  scrutinize  the  various  roles  played  by  sound  in  a  3-dimensional  space,  we 
need  to  record  various  kinds  of  sound  and  arrange  them  over  a  3-dimensional 
space.  In  the  following  paragraphs,  the  writer  will  describe  a  method  for 
arranging  and  manipulating  sound  in  a  3-dimensional  space. 

To  implement  such  a  sound  environment,  sound  is  handled  3-dimensionally 
using  virtual  reality  techniques.  To  make  it  easy  to  manipulate  sounds,  they 
are  represented  as  metaphors,  allowing  them  to  be  dealt  with  directly  in  a  3- 
dimensional  space.  These  metaphors  reflects  the  properties  of  sounds  or  the 
shape  of  the  sound  sources.  Once  a  sound  has  been  converted  into  a  metaphor, 
it  can  be  manipulated  inside  a  3-dimensional  space.  To  build  up  this  system, 
tool  and  material  metaphors  and  the  concept  of  chambers  have  been  employed. 
The  chambers  consist  of  a  recording  chamber,  a  shape-processing  chamber 
and  an  experiential  chamber.  In  the  recording  chamber,  sound  is  recorded 
and  a  shape  corresponding  to  the  attributes  of  the  sound  is  created.  In  the 


shape-  processing  chamber,  a  shape  conforming  to  the  sound  source  is 
produced.  Thus,  a  sound  metaphor  is  completed.  In  the  experiential  chamber, 
the  sound  metaphor  is  arranged  over  a  3-dimensional  space,  so  that  a  3- 
dimensional  type  of  sound  can  be  experienced.  This  system  is  built  around  a 
graphics  workstation  and  dedicated  virtual  reality  (VR)  equipment.  The 
recording  chamber  and  shape-  processing  chamber  are  located  within  the 
graphics  workstation,  and  the  experiential  chamber,  within  the  dedicated  VR 
machine. 

[Miwa  et  al  (Matsushita  Electric)  emphasized  the  importance  of  sound 
manipulation  in  3D  space,  as  a  step  beyond  current  typical  document 
processing  tasks.  The  problems  encountered  when  trying  to  work  in  this 
manner  with  sound  are  that  sound  is  invisible,  difficult  to  manipulate,  and 
difficult  to  position  in  3D  space.  The  objective  of  the  Matsushita  project  is  to 
"metaphorise"  sound  using  a  chamber  metaphor,  where  sounds  are  treated  as 
materials  that  are  manipulated  by  a  set  of  tools  in  successive  chambers.  These 
latter  include  a  recording  chamber,  a  shape  processing  chamber,  a  trajectory 
generating  chamber  and  an  experiential  chamber.  The  tools  comprise  means 
of  changing  sound  volume,  transforming  and  separating  shapes,  deleting 
sounds,  and  altering  acoustics.  The  final  sound  icons  are  presented  visually 
and  manipulated  within  a  virtual  reality  environment.] 


A  Metaphor  Environment  Based  on  Interactive  Scenarios 

Masanobu  Sakaguchi,  Keiko  Sakuragi,  Toshiro  C.  Watanabe 
Corporate  Research  Laboratories 
Sony  Corporation 

Metaphor  environment  is  a  kind  of  human-computer  interface,  and  is 
constructed  with  some  suitable  metaphors  for  context.  And  it  has  a  feature  of 
showing  the  current  operation  contents  and  status  of  a  computer  system  to  its 
users  in  the  contextual  and  understandable  way  (WAT90). 

We  have  investigated  metaphor  environment  (SAK93),  and  proposed  a 
framework  of  the  environment  which  consists  of  three  major  elements  -- 
Hermeneutics,  Image,  and  Task  (Fig.  1)  (WAT93);  the  framework  is  named  the 
HIT  model  after  those  elements.  It  represents  the  relation  that  a  task  can  be 

mapped  to  an  image  by  certain  interpretation  (hermeneutics),  and  shows  a 
method  how  the  current  operation  and  status  of  the  computer  can  be 
represented  with  an  image  which  is  easy  to  understand  by  the  user. 

In  our  framework,  "Hermeneutics"  is  the  basis  on  which  a  task  (user's  tenor) 
can  be  represented  by  an  image  (vehicle)  as  a  metaphor.  In  an  application 
system,  the  whole  of  interpretation  is  a  kind  of  world,  and  has  a  role  of  guiding 
the  user  based  on  scenarios  which  have  consistent  stories.  We  have  proposed 
that  such  a  metaphor  environment  as  a  human-computer  interface  should  be 
built  based  on  the  scenarios. 

People  who  design  and  build  the  metaphor  environment  are  system  designers, 
and  people  who  work  in  the  environment  are  users.  If  there  is  a  large  gap 
between  user-hermeneutics  assumed  by  the  designer  and  user's  actual 
hermeneutics,  the  user  feels  that  the  human  interface  of  the  computer  is  hard 
to  understand  and  difficult  to  use.  However,  it  is  impossible  to  make  the 
interface  in  which  designer-assumed  hermeneutics  and  user's  hermeneutics 
are  completely  the  same. 


For  this  reason,  we  think  that  we  should  consider  users'  point  of  view  more 
importantly,  and  devised  a  repeated-design  method  for  a  flexible  metaphor 
environment,  with  which  the  environment  can  adapt  itself  more  suitable  for 
the  user  while  he/she  uses  an  application  program  in  the  environment.  Since 

understandability  and  usability  are  more  important  for  computer  beginners 
than  experienced  users,  we  focus  on  the  beginners.  From  now  on,  we  use  the 
word  "user"  for  the  computer  beginners. 

In  Chapter  2,  we  first  describe  the  metaphor  environment  from  user's 
viewpoint  and  discuss  the  user  world  which  the  system  designer  should 
assume.  And  we  describe  a  design  method  of  a  virtual  world  in  which  users 
perform  their  tasks.  Next,  we  illustrate  our  system  called  "Macro  Browser" 

(OHB  93)  as  an  example  of  the  method  in  Chapter  3.  Then,  in  Chapter  4,  we 

explain  an  experiment  on  Macro  Browser  and  give  the  evaluation  of  this 
system  from  a  viewpoint  of  the  environment's  helpfulness  and  effectiveness 
to  user's  understanding  and  task  performance.  Finally,  we  conclude  this  paper 
with  the  importance  of  the  framework  of  the  metaphor  environment  which 
we  proposed. 

[Sakaguchi  et  al  (Sony)  described  their  work  on  metaphor  environments  based 
on  interactive  scenarios,  where  scenarios  are  used  as  a  means  of  presenting  to 
a  user  the  correspondence  between  a  series  of  tasks  and  a  chain  of  pictures. 
The  goal  is  to  teach  the  user  about  possible  actions  permitted  by  the  system 
through  constraints,  which  are  based  on  context.  The  metaphor  environment 
is  created  iteratively  by  designing  task  transitions  which  make  the  user's 
objective  possible,  writing  scenarios  which  express  the  task  model  to  the  user, 
and  creating  images  based  on  scenarios  that  match  each  task.  The  authors' 

system  for  doing  this  is  Macro  Browser,  which  allows  video  viewing,  creation, 
editing,  playing,  browsing,  etc.,  all  while  recording  the  user's  operating 

history.] 


Visualizing  Information  by  Metaware 

Nobuyuki  Makimura,  Fujitsu  Laboratories  Ltd. 

Eiichiro  Yamamoto,  Fujitsu  Limited 

In  the  21®‘  century,  it  is  said  we  shall  enter  upon  highly  personalized 
information  society.  In  its  personal  information  environment,  we  shall  be 

able  to  handle  multimedia  data. 

In  the  FRIEND21  project,  we  were  studying  the  Hypermedia  Authoring  System 
as  one  of  tools  which  help  almost  all  people  express  personal  information.  In 
that  study,  we  supposed  business  presentation  as  one  of  current  applications  of 

hypermedia  authoring  systems.  We  defined  three  casts  in  presentation,  the 

author  who  prepares  the  presentation  materials,  the  performer  who  presents 
it,  and  the  audience  who  observes  it.  We  considered  what  metaphor  is 
appropriate  for  each  cast  and  what  metaphor  can  be  easily  understood 

according  to  the  difference  of  presentation  environment  and  so  on  (1). 

From  this  consideration,  we  reached  the  recognition  that  it  is  also  important  to 
express  contents  of  information  by  some  metaphors. 

Meanwhile,  in  the  FRIEND21  project,  it  has  been  mainly  discussed  how  we  can 
express  functions  or  operations  of  the  computer  by  using  metaphor 
environment  for  the  purpose  of  helping  people  easily  understand  these 
functions  or  operations.  That  is,  the  main  subject  there,  is  to  help  people 


understand  the  computer  operation  to  have  access  to  or  process  or  create 
information. 

But,  there  exist  not  only  operations  but  also  information  in  the  computer.  Of 
course,  it  is  important  to  understand  the  operations  to  have  access  to 
information  etc.,  but  it  is  also  important  to  understand  information  itself.  As 

information  is  abstract,  it  will  become  to  be  easily  understood  when  it  has 
concrete  shape.  For  example,  to  understand  the  structure  of  an  atom,  it  is 
useful  to  suppose  its  structure  as  that  of  the  solar  system.  So  we  think  that 
metaphors  are  effective  also  in  case  of  expression  of  information. 

In  this  paper,  we  consider  metaphors  which  give  forms  to  information  and 
which  help  people  understand  information. 

[Makimura  and  Yamamoto  (Fujitsu)  dealt  with  the  important  problem  of 
visualizing  information  by  metaware,  that  is,  how  to  present  information 
visually  to  users  in  an  understandable  fashion.  To  understand  task  information 
easily,  we  have  to  be  able  to  answer  questions  encompassing  What,  Who,  Why, 
Where,  When  and  How.  This  particular  talk  concentrated  on  the  why,  where 
and  when  questions,  which  comprise  such  issues  as  comparisons,  structures, 
relations,  classifications  and  viewpoint.  Their  system  uses  the  IntelligentPad 
on  Interviews  to  generate  views  and  allow  dynamic  changes  in  the  way 
information  is  viewed  and  presented.] 


Coordinating  an  interface  agent  with  direct  manipulation  environments 
Takashi  Sonoda 
Foundation  Res.  Lab. 

Fuji  Xerox  Co.,  Ltd. 

In  this  paper,  we  propose  a  collaborative  manipulation  interface  which 
includes  an  interface  agent  as  the  "dialogue  partner"  and  direct  manipulation 
interface  as  the  "tools."  These  tools  are  shared  and  collaboratively 
manipulated  by  the  user  and  the  agent.  It  is  thought  that  this  interface  system 
simulates  human  cooperative  works.  This  interface  has  the  same  properties  of 
the  cooperative  works.  We  describe  the  benefits  of  the  collaborative 

manipulation  interface.  A  prototype  system  for  the  group  schedule 

management  is  also  developed. 

[Sonoda's  (Fuji  Xerox)  paper  dealt  with  coordinating  interface  agents  with 
direct  manipulation  environments.  Direct  manipulation  interfaces  are  those 
in  which  procedural  knowledge  lies  with  the  user  and  the  interface  is  viewed 
as  a  "tool".  Intelligent  agent  type  interfaces,  on  the  other  hand,  are  those  in 
which  the  agent  acts  as  a  dialogue  partner  who  understands  the  intentions  of 
the  user.  Rather  than  using  exclusively  either  one  or  the  other,  this  paper 
proposed  coordinating  the  use  of  both  approaches  collaboratively,  as  a  means 
of  enhancing  co-operative  work.  In  this  complementary  approach,  the  user 
learns  from  the  agent/coworker,  and  gradually  takes  over  the  task  as 
knowledge  is  acquired  and  as  task  circumstances  dictate.] 


Multiple-Metaphor  Environment 

Makoto  Hirose 

Systems  &  Communications  Lab. 

Fuji  Xerox  Co.,  Ltd. 

One  of  the  most  important  issues  discussed  in  this  paper  is  whether  interface 
metaphors  will  be  abandoned  thorough  their  repeated  use.  Instead  of  using  a 
mouse  or  icons,  we  may  type  commands  on  keyboards  directly;  "read"  files, 
"kill"  windows,  "arrest"  processes.  It  seems  that  such  interface  metaphors 
have  been  abandoned.  However,  such  commands  do  not  literally  refer  to 
killing  or  arresting  anyone.  These  kinds  of  expressions  are  merely  interface 
metaphors  that  assist  a  user  in  understanding  and  experiencing  what  is 
happening  inside  the  computer  (1). 

Interface  metaphors  are  tools  that  help  people  understand  and  use  computers. 
Given  that  people  will  utilize  information  media  in  almost  all  walks  of  life  in 
the  century,  we  propose  that  interface  metaphors  should  be  able  to 

accommodate  a  combination  of  interface  metaphors  flexibly,  freely  and 
dynamically  in  the  same  way  that  people  can  combine  a  finite  number  of 
words  in  a  wide  variety  of  ways  to  express  their  feelings.  The  metaware 
mentioned  below  offers  a  methodology  for  combining  interface  metaphors 
flexibly,  freely,  and  dynamically. 

Metaware  aims  at  creating  a  Multiple-Metaphor  Environment.  The  Multiple- 
Metaphor  Environment  can  process  both  "synonymy"  and  "polysemy,"  which 
cannot  be  processed  by  most  existing  interface  metaphors.  Also,  it  possesses 
context  sensitivity,  which  means  that  it  can  utilize  both  "synonymy"  and 
"polysemy"  in  a  positive  way.  At  the  same  time,  its  time  phase  function  is 
based  on  cognitively  tractable  scenarios. 

[In  an  extended  lecture,  Hirose  (Fuji  Xerox)  talked  about  the  need  for  multiple 
metaphor  environments  (MME),  the  potential  roles  they  can  play,  and  some 
issues  involved  in  putting  them  into  practice.  One  of  the  problems  with 
current  interfaces  is  that  most  metaphors  are  context  sensitive  and  their 
appropriateness  can  change  as  tasks,  computer  functions  and  users'  levels  of 
knowledge  change.  This  presents  a  problem  to  designers  about  how  to  decide 
in  advance  what  are  optimal  interface  metaphors.  Multiple  Metaphor 
environments  address  this  by  providing  synonymy  /  polysemy,  multiplicity 
and  context  sensitivity,  all  within  a  common  interface.  The  principal  message 
of  Hirose's  talk  was  an  interesting  analysis  of  multiple  metaphor  cases  in 
terms  of  a  two-dimensional  taxonomy,  in  which  polysemy  and  synonymy  trade 
off  against  each  other  along  one  continuum  and  spatiality  and  temporality 
along  the  other  continuum.  Outstanding  issues  include  how  to  combine 
customization  with  identification,  in  order  to  provide  the  user  with  the 
preferred  interface.] 


Agents  and  Information  Environments  Presentations 


Leaning  Interface  Agents 
Pattie  Maes 
MIT  Media-Lab 
20  Ames  Street  Rm.  401a 
Cambridge,  MA  02139 


Interface  agents  are  computer  programs  that  employ  Artificial  Intelligence 
techniques  in  order  to  provide  assistance  to  a  user  dealing  with  a  particular 
computer  application.  The  paper  discusses  an  interface  agent  which  has  been 
modeled  closely  after  the  metaphor  of  a  personal  assistant.  The  agent  learns 
how  to  assist  the  user  by  (i)  observing  the  user's  actions  and  imitating  them, 

(ii)  receiving  user  feedback  when  it  takes  wrong  actions,  (iii)  being  trained 
by  the  user  on  the  basis  of  hypothetical  examples  and  (iv)  learning  from  other 
agents  that  assist  other  users  with  the  same  task.  The  paper  discuss  how  this 
learning  agent  was  implemented  using  memory-based  learning  and 
reinforcement  learning  techniques.  It  presents  actual  results  from  two 
prototype  agents  built  using  these  techniques:  one  for  a  meeting  scheduling 
application  and  one  for  electronic  mail.  It  argues  that  the  machine  learning 
approach  to  building  interface  agents  is  a  feasible  one  which  has  several 
advantages  over  other  approaches:  it  provides  a  customized  and  adaptive 
solution  which  is  less  costly  and  ensures  better  user  acceptability. 

[Maes  (MIT  Media  Lab)  gave  an  interesting  account  of  work  on  intelligent 
agents  modeled  according  to  the  metaphor  of  a  personal  assistant  / 
collaborator.  Such  systems  are  expected  to  become  more  common  as  the  need  to 
augment  the  direct  manipulation  metaphor  increases.  Two  important  problems 
with  such  systems,  however,  are  competence  and  trust.  Two  approaches  to 
building  interface  agents  were  identified.  In  one  class,  the  knowledge 
engineer  programs  the  end-user  program,  which  thus  becomes  the  agent  and 
is  used  at  run  time  to  infer  user’s  goals;  that  is,  it  acts  between  the  user  and  the 
application.  The  other  (knowledge-based)  approach,  comprises  an  interface 
agent  with  a  very  large  amount  of  fixed  domain-specific  knowledge  about  the 
application  (domain  model)  and  the  user  (user  model).  At  run  time  the 
interface  agent  uses  this  knowledge  to  recognize  the  user's  plans  and  provide 
assistance.  Neither  approach  solves  either  the  competence  problem  or  the 
trust  problem,  since  end  users  do  not  want  to  program  the  applications  and 
since  the  knowledge  engineer  is  not  dealing  with  the  end  user.  The  Media  Lab 
approach  involves  learning  interface  agents,  which  are  similar  to  apprentices 
who  are  given  some  background  knowledge  but  acquire  most  of  their 
knowledge  from  observing  the  user,  from  being  explicitly  trained,  and  from 
consulting  with  other  agents  which  have  more  experience.  The  examples 
presented  included  a  personal  assistant  for  meeting  scheduling,  a  personal 
assistant  for  electronic  mail,  a  personal  editor  for  (electronic)  news,  and  a 
personal  agent  for  recommending  books,  movies,  music,  etc.  Experiences  with 
these  systems  thus  far  are  quite  promising,  from  the  points  of  view  of  both 
competence  and  trust.] 


Towards  Intimate  Interaction 
Mario  Tokoro 

Keio  University  and  Sony  Computer  Science  Laboratory,  Inc. 

Akkikazu  Takeuchi 

Sony  Computer  Science  Laboratory,  Inc. 

Intimacy  means  secure,  piece  of  mind,  trustworthy,  reliable,  and  respect. 
Intimate  interaction  is  the  new  notion  of  human-computer  interaction,  which 
is  a  basis  of  new  human-computer  interaction  giving  users  such  feeling.  In 
the  near  future,  computers  become  everyday  tools  that  make  our  life 
convenient  and  comfortable.  They  will  ;be  used  as  an  access  terminal  to 


various  networked  services,  and  will  be  used  as  a  communication  terminal  to 
other  computers  and  humans.  But  they  will  be  used,  consciously  or 
unconsciously,  just  as  a  conversation  partner.  Namely,  the  ultimate 
application  is  the  dialogue  itself.  In  this  interaction,  intimacy  is  essential  to 
help  people  believe  a  computer  as  an  individual  agent.  Through  such 
interaction,  people  will  believe  a  computer  as  an  individual  agent.  Through 

such  interaction,  people  will  get  the  feeling  of  recognizing  each  other, 
understanding  each  other,  and  sharing  common  experience.  In  order  to 
concretize  the  notion  of  intimate  interaction,  we  have  many  research  items  to 

challenge.  Among  them,  as  a  key  technology,  we  pay  special  attention  to 
multimodal  interaction,  especially  speech  dialogue  with  facial  displays.  The 

human  face  is  an  independent  communication  channel  that  conveys 
emotional  and  conversational  signals  encoded  as  facial  displays.  We  are 
attempting  to  introduce  facial  displays  into  multimodal  human  computer 
interaction  as  a  new  modality  to  make  computer  more  communicative  and 

social.  As  a  first  step,  we  developed  a  multimodal  human  computer  interaction 
system  integrating  speech  dialogue  and  facial  animation.  The  paper  gives  an 
overview  of  the  notion  of  intimate  interaction,  and  then  describes  the  speech 
dialogue  with  facial  displays  system  focusing  on  major  research  topics.  They 
are  to  understand  and  manage  speech  dialogue,  to  design  and  animate 
communicative  facial  displays,  and  to  combine  multiple  modalities,  that  is, 
speech  and  facial  displays. 

[Tokoro  (Keio  University  &  Sony)  gave  a  rather  philosophical  talk  on 
"intimate  interaction"  between  humans  and  computers,  which  refers  to  the 
scenario  in  which  the  computer  will  be  a  ubiquitous  tool  with  whom  the  user 

will  share  a  sense  of  trust,  security,  reliance,  and  respect.  (This  scenario  was 
illustrated  by  means  of  an  entertaining  video.)  Tokoro’s  future  cohabiting 
society  of  autonomous  computers  and  humans  encompasses  a  Computational 
Field  Model  comprising  a  continuously  fluctuating  sea  of  ubiquitous 
intercommunicating  computers,  in  whose  "computational  field"  objects  "float" 
and  are  subject  to  real-time  forces  analogous  to  gravity  and  inertia.  People  will 
always  carry  with  them  their  "Intimate  Computer",  which  will  act  as  an 

interface  terminal  with  the  Computational  Field.  One  concrete  aspect  of 

multimodal  human  interaction  with  an  autonomous  volitional  agent  is  the 
combination  of  the  modalities  of  speech  dialogue  and  facial  displays,  to  create 

an  interface  in  which  the  computer  actually  converses  with  the  user. 

Examples  were  shown  of  the  Sony  face  display  dialogue  system.] 


"Forget-me-not" 

Intimate  Computing  in  Support  of  Human  Memory 
Mik  Lamming  and  Mike  Flynn 
Rank  Xerox  Research  Center 

At  RXRC  we  have  been  trying  to  understand  how  anticipated  developments  in 
mobile  computing  will  impact  our  customers  in  the  2U*  century.  One 
opportunity  we  can  see  is  to  improve  computer-based  support  for  human 
memory  -  ironically  a  problem  in  office  systems  research  that  has  almost  been 
forgotten.  Considering  how  often  computers  are  presented  as  devices  capable 
of  "memorizing"  vast  quantities  of  information,  and  performing  difficult-to- 
memorize  sequences  of  operations  on  our  behalf,  we  might  be  surprised  at  how 
often  they  appear  to  have  increased  the  load  on  our  own  memory. 


The  Forget-me-not  project  is  an  attempt  to  explore  new  ways  in  which  mobile 
and  ubiquitous  technologies  might  help  alleviate  the  increasing  load.  Forget- 

me-not  is  a  memory  aid  designed  to  help  with  everyday  memory  problems: 
finding  a  lost  document,  remembering  somebody's  name,  recalling  how  to 
operate  a  piece  of  machinery.  It  exploits  some  well  understood  features  of 
human  episodic  memory  to  provide  alternative  ways  of  retrieving  information 
that  was  once  known  but  has  now  been  forgotten. 

We  start  by  introducing  a  model  of  computing  in  the  2P*  century  which  we 
call  the  "Intimate  Computing"  model  and  talk  about  some  of  the  opportunities 
and  problems  we  anticipate  it  will  provoke.  After  a  cursory  introduction  to  the 
basics  of  human  episodic  memory,  we  describe  the  architecture  and  user 
interface  of  Forget-me-not.  We  end  with  a  few  preliminary  conclusions  drawn 
from  our  early  experiences  with  the  prototype. 

[Lamming  (Rank  Xerox  Research  Centre)  gave  an  entertaining  talk  on 
intimate  computing  in  support  of  human  memory.  He  pointed  out  the 
anticipated  need  for  a  cheap  reliable  memory  aiding  device,  as  a  future 
computing  device  which  will  have  a  market  similar  to  that  of  today's  digital 
wristwatch.  The  next  revolution  (analogous  to  Tokoro's  vision)  will  be  in 
mobile  computing,  and  will  comprise  small  computers  which  can  be  worn, 
which  will  communicate  with  each  other  via  cellular  communication,  and  will 
always  know  where  they  are.  Work  underway  at  EUROPARC  is  centered  on 
trying  to  understand  what  the  world  will  be  like  when  everyone  has  an 
intimate  computer  on  her  wrist.  For  example,  we  will  be  able  to  exchange  data 
with  other  people  at  any  time,  including  during  chance  meetings  on  the 
street;  we  will  be  able  to  collect  any  data  that  we  need,  as  we  move  around  at 
any  time;  we  will  always  have  help  in  finding  places,  phone  numbers,  talk 
slides,  etc.  Our  Personal  Data  Assistants  (PDA's)  will  have  access  not  only  to  our 
data  bank,  but  also  to  our  context  ,  and  thus  will  able  to  create  episodic 
memory,  by  maintaining  a  list  of  episodes  and  generating  scenarios  based  on 
this  to  allow  probing  of  episodes.  One  exploratory  area  at  Xerox  is  the  Active 
Badges  worn  by  all  there,  which  are  monitored  by  sensors  all  over  the 
building,  causing  all  movements  to  be  stored  in  a  log  file.  The  kinds  of  things 
being  learned  include  privacy  and  security  issues,  the  difficulty  of  getting  lost 
in  the  database,  the  need  for  reliability  (memory,  etc.),  the  need  to  improve 
performance  (speed),  the  difficulties  involved  in  translating  raw  data  into 
events  that  the  user  will  recognize,  and  the  social  implications  of  having 
"perfect  memory".] 


Agency  Model  Presentations 


Agency  Model  for  Acoustic  Multimodal  Interaction 
Hidai  Yutaka 

FRIEND21  Research  Center 

We  have  proposed  an  Agency  Model  based  on  the  Blackboard  Model  as  human 
interface  architecture  for  the  2D‘  century.  This  paper  describes  a  new 
attempt  to  add  a  framework  for  acoustic  human  interface  to  the  Agency  Model 
At  present,  the  sense  of  sight  is  used  in  the  most  advanced  manner  for  human 
interface.  However,  the  sense  of  hearing  is  just  as  important  as  the  sense  of 
sight  for  interaction  with  other  human  beings. 


To  make  interactions  more  understandable,  multimodal  interaction 
technologies  using  multiple  communication  media/modes  can  be  very  helpful. 
Communication  using  a  medium  can  include  multimodal  interactions.  Another 

factor  that  improves  comprehension  for  humans  is  transparency:  internal 

data  should  be  shown  to  users  in  a  directly  visible  form.  In  addition,  the 
processing  of  modes  is  important  in  multimodal  interaction. 

In  acoustic  multimodal  interaction,  it  is  important  to  synchronize  the 
processing  of  various  modules  since  time  components  are  contained  in 
respective  communication  modes.  We  propose  to  perform  synchronization  by 
logical  time,  which  is  handled  individually  in  each  module. 

As  a  subject,  this  research  uses  music  as  an  example,  which  is  a  kind  of 
acoustic  multimodal  interaction.  This  study  is  intended  to  solve  problems 
involving  acoustic  human  interface,  multimodal  interaction,  synchronization, 
and  internal  expression  similar  to  physical  space  sound.  Last  year,  we  studied 
the  concept  of  logical  time,  and  created  an  experimental  system  WALTZ  which 
generated  a  single  melody  using  logical  time. 

This  paper  describes  an  accompaniment  system  that  uses  internal  acoustic 
expression  and  time  management  concepts  in  multimodal  interaction  based  on 
the  logical  time  concept.  In  this  system  the  studio  is  regarded  as  a  space  and 
data  are  written  as  sounds  in  the  studio.  Synchronization  is  performed 
according  to  differences  with  the  other  players. 


A  Cooperative  Mechanism  for  Acoustic  Multimodal  Interaction 

Takashi  Sugai 

FRIEND21  Research  Center 

Institute  for  Personalized  Information  Environment 

We  consider  distributed  cooperative  architecture  such  as  the  Agency  Model  to 
be  suitable  for  multimodal  processing.  This  paper  proposes  a  cooperative 
mechanism  for  acoustic  multimodal  interaction  and  describes  its 
implementation  based  on  the  Agency  Model.  The  cooperative  mechanism  is 
derived  from  a  model  of  acoustic  multimodal  interaction.  We  regard  acoustic 
multimodal  interaction  as  mainly  consisting  of  three  major  modes:  sound 
length  (tempo),  frequency,  and  amplitude.  This  research  was  conducted  to 

search  for  an  interaction  mechanism  when  agents  in  the  Agency  Model  are 
assigned  to  the  communication  modes.  To  simplify  the  issue,  this  study  is 
limited  to  certain  modes  in  terms  of  sound  length.  As  an  example,  a  music 
accompaniment  system  has  been  designed  and  built  based  on  the  cooperative 
mechanism.  The  accompaniment  system  shows  that  the  cooperative 
mechanism  is  effective  under  the  following  circumstances:  When  a  mode  that 
is  not  anticipated  by  the  system  exists  on  the  soloist  side,  when  knowledge  is 
added  to  the  soloist  side,  and  when  an  agent  is  plugged  into  the  accompaniment 
system,  because  each  agent  need  not  explicitly  consider  to  which  agent  a 
transfer  should  be  made. 


"PAINT  IT  BLACK" 

-  A  Multimodal  Drawing  Tool  - 
Mayumi  Hiroshi 
NEC  Corporation 


A  multimodal  user  interface  allows  users  to  communicate  with  computers  by 
using  multiple  modalities,  such  as  a  mouse,  a  keyboard,  or  a  voice  input,  in 
various  combined  ways.  In  such  a  system,  individual  input/output  modes  have 
to  work  independently  and  cooperatively.  Agency  Model  proposed  by 
FRIEND21  Project  is  a  framework  where  several  independent  agents  work 
cooperatively  (1).  So  it  is  suitable  as  an  architecture  for  a  multimodal 
interface.  This  paper  discusses  a  multimodal  interface  for  a  drawing  tool, 
especially  focusing  on  multimodal  inputs.  First,  the  author  describes  the 
multimodal  interface  positioning  used  in  the  study,  as  well  as  multimodal 
inputs  in  a  drawing  tool.  Next,  a  prototype  system,  developed  based  on  Agency 
Model,  and  it’s  multimodal  inputs  interpretation  method  are  described. 


Information  retrieval  process  based  on  structured  data 
Makoto  Nishida 
Dai  Nippon  Printing  Co.,  Ltd. 

A  problem  of  Human  Interface  for  retrieving  multimedia  information,  e.g. 
music,  photograph,  and  movie,  are  complicated  methods  for  retrieving.  At 
present,  methods  for  retrieving  multimedia  information  are  studied 
individually  in  each  media.  To  retrieve  simultaneously  multimedia  information 
in  the  present  state  of  things,  we  expect  that  methods  for  retrieving  are 
difficult,  and  we  cannot  find  out  easily  needed  information. 

To  make  retrieval  easy,  it  is  necessary  to  be  able  to  retrieve  multimedia 
information  by  common  methods.  For  the  purpose,  multimedia  information 
should  be  represented  by  single  media.  We  must  categorize  them.  And  we  must 
study  how  to  utilize  them  to  retrieve  easily  multimedia  information. 

For  doing  so,  we  aimed  to  represent  multimedia  information  by  texts,  and  to 
retrieve  them  via  the  texts.  First,  we  analyzed  retrieval  methods  based  on  texts, 
e.g.  dictionary.  We  extracted  structures  of  information  for  retrieval  that  users 
aren't  conscious  of  differences  in  media,  and  categorized  them.  The  structured 
information  were  represented  by  texts.  Secondly,  we  studied  the  mechanism 
that  users  can  freely  choose  an  appropriate  function  in  response  to  a 
transition  of  the  user's  states  in  retrieval  processes. 

In  this  paper,  we  suggest  retrieval  processes  based  on  structured  texts  data  for 
multimedia  information  retrieval.  We  describe  a  prototype  system  based  on 
the  new  concept,  and  a  experiment  on  the  system. 


Information  Structure  of  User  Interface  for  Integration  of  Text  and  Images 
Shogo  Nakagome 
Toppan  Printing  Co.,  Ltd. 

Processing  manuscripts  on  computer  systems  has  been  starting  point  for 
many  of  us  to  become  a  computer  user. 

Document  processing  is  one  of  the  most  typical  and  popular  forms  of  computer 
usage  for  both  novice  and  expert  users.  It  is  very  important  to  provide  many 
users  with  comfortable  environment  for  document  processing. 

For  document  to  be  effective  in  conveying  information,  the  system  to  process 
documents  has  to  integrate  text  and  images.  Word-processors  are  one  of 
typical  electronic  document  processing  systems.  Recently,  they  provide 
additional  functions  for  processing  graphics  and  spreadsheets.  However,  word 


processors  currently  available  often  lack  functionalities  sufficient  to  handle 
editing  and  layout  when  processing  text  and  graphics  data. 

An  environment  comfortable  for  a  user  is  in  a  subtle  way  different  from  the 
one  for  another.  Because  each  user  has  different  value  system,  knowledge 
background,  and  purpose  of  usage.  However,  all  users  prefer  to  have  the 
environment  where  they  can  make  documents  without  being  troubled  by 
different  operations  between  text  and  images. 

What  users  need  now  is  the  environment  enabling  the  integration  of  text  and 
images  in  a  natural  form. 

This  paper  describes  a  definition  of  information  structure  to  integrate  both 
text  and  images,  and  its  implementation  as  well. 


Implementation  of  a  User  Adaptation  Mechanism  on  the  Agency  Model 
Shigeo  Sumino,  Takafumi  Miyatake,  Hirotada  Ueda 
Central  Research  Laboratory,  Hitachi,  Ltd. 

With  recent  progress  in  computer  science,  people  have  become  familiar  with 
computer  systems.  For  example,  home  computer  games  are  treated  like 
domestic  electronic  goods.  However,  the  usability  of  these  systems  varies.  In 
some  recent  applications,  the  button  assignments  can  be  changed  by  the  user 
because  his  viewpoint  may  be  different  from  that  of  the  application  designer. 
The  FRIEND21  project  of  the  Japanese  Ministry  of  International  Trade  and 
Industry  has  been  promoting  research  in  the  field  of  Human-Computer 
Interaction  since  1988.  For  example,  Metaware  (7)  and  the  Agency  Model  (6) 
are  its  outcome.  As  part  of  this  project,  we  have  engaged  in  a  research  on  an 
adaptive  interface  that  can  be  changed  according  to  the  user  preference  in 
order  to  improve  its  usability. 

Rich  proposed  a  basic  idea  for  this  interface,  that  is  to  transform  the  canonical 
model  according  to  the  individual  user  operations  (5).  Croft  explained  the 
adaptive  method  according  to  the  user's  operational  context  (1).  Greenberg  et 
al.  proposed  the  system  that  detects  the  most  frequently  used  menu  by  utilizing 
a  tree  structure  and  the  frequency  of  usage,  and  then  transforms  the  menu 
structure  (3).  In  this  method,  the  menu  structure  is  transformed  dynamically 
according  to  user  operations,  but  objects  operated  on  by  the  menu  are  not 
considered.  Cypher  reported  a  prototype  system.  Eager  (2),  run  on  Macintosh's 

HyperCard  system.  Eager  monitors  user  operations,  detects  repetitions,  and 
offers  to  automate  them.  Eager  considers  the  menu  operation,  but  does  not 
handle  sequences  that  are  not  repeated  including  mistaken  operations. 

In  this  paper,  we  propose  a  user  adaptation  mechanism  that  consists  of  two 
adaptive  methods  and  show  our  prototype  implementation  of  these  methods. 
One  method  which  deals  with  functional  sequences  takes  into  account  objects 
operated  on  by  functions  and  user  context.  The  other  method  deals  with 
functional  parameters. 

In  Section  2,  we  propose  a  user  adaptation  mechanism  consisting  of  the  above 
two  methods.  ;In  Section  3,  we  give  the  specifications  of  prototype  that 
implements  two  adaptation  methods  for  a  motion  picture  editing  system.  In 
Section  4,  we  discuss  problems  of  the  prototype  and  explain  improvements  to 
the  system.  In  Section  5,  we  summarize  results. 


User  adaptation  by  task  intention  identification 
Kazuhisa  Odaka 
FRIEND21  Research  Center 

Institute  for  Personalized  Information  Environment 

Even  users  with  an  understanding  of  the  internal  processes  of  a  computer 
system  may  not  understand  how  to  operate  such  a  system  most  efficiently. 
Therefore,  many  researchers  have  pointed  out  the  importance  of  a  metaphor 
interface  that  provides  a  conceptualized  model  of  the  system's  operation, 
thereby  drawing  upon  a  user's  current  knowledge  and  skills  (1)(2). 
Nevertheless,  a  metaphor  interface  is  not  without  its  problem  so  if  the  user 
develops  a  concept  from  the  metaphor  that  is  different  from  the  designer's 
original  intention  the  system  may  not  function  as  expected,  and  may  therefore 
confuse  the  user. 

Such  problems  arise  from  a  lack  of  understanding  about  what  the  metaphor 
means  to  the  user.  The  metaphor  interface  involves  mapping  between  the 
functions  of  an  application  and  the  visual  objects  of  the  metaphor.  Most 
problems  are  caused  by  improper  mapping. 

Moreover,  a  user's  actions  may  change  depending  on  the  situation.  Normally,  a 
computer  is  used  either  to  acquire  or  express  knowledge.  The  user  performs  a 
sequence  of  operations  to  extract  this  knowledge.  As  such,  different  task 
intentions  require  different  sequences  of  individual  operations. 

A  computer  system  that  can  take  into  consideration  a  user's  task  intentions  can 
support  the  particular  methods  he  uses  intuitively.  The  user  is  thus,  free  to 
concentrate  on  the  task  itself  without  being  distracted  by  other  concerns. 

The  aim  of  this  research  is  to  allow  the  mapping  to  be  adapted  to  each  situation 
through  the  use  of  a  mechanism  that  identifies  the  user's  situation, 
particularly  the  task  context.  In  this  way,  the  system  can  eliminate 
incomplete  mapping  between  application  functions  and  metaphor  objects. 

We  believe  that  it  is  possible  to  obtain  part  of  the  task  context  from  the  user's 
operation  history.  The  prototype  we  have  developed  makes  decisions  about 
which  function  to  perform  based  on  the  task  contexts  from  the  user's 
operation  history.  Using  this  system,  we  examined  the  mapping  mechanism  to 
see  if  it  would  work  properly. 


Filtering  Software  Products  and  Advertising  Software  Products: 

Two  Sides  of  the  Same  Coin? 

Masashi  Uyama 

FRIEND21  Research  Center 

Software  products  provide  many  software  services  to  support  users'  task 
executions.  In  an  open  network  environment,  where  newly  designed  software 
products  are  frequently  disseminated  and  installed,  computer  systems  should 
help  users  find  useful  software  services  and  integrate  such  services  into  their 
tasks.  This  paper  proposes  a  three-step  filtering  mechanism  to  support  a  user's 
mental  process  through  which  the  user  passes  from  first  awareness- 
knowledge  of  new  software  services  to  a  decision  about  adopting  or  rejecting 
such  services.  The  filtering  mechanism  selects  services  that  trustworthy 
colleagues  have  recommended,  and  then,  selects  services  specific  to  the 
context  of  the  user's  task  executions.  Finally,  the  mechanism  discloses  the 
selected  services  to  the  user  dynamically  and  unobtrusively.  The  dynamic 


disclosure  allows  users  to  try  out  new  services  in  their  own  task  context.  The 
disclosure  is  unobtrusive  since  users  can  ignore  the  disclosure  and  continue 
their  current  tasks  without  learning  any  of  the  new  features.  The  three-step 
filtering  mechanism  has  been  implemented  based  on  the  Agency  Model. 

6.  Sponsor's  Demonstration  Projects 

A  total  of  fifteen  demonstration  projects  were  available  for  view  during 
scheduled  interactive  sessions  on  each  of  the  three  days  of  the  symposium.  All 
of  the  demonstration  projects  concern  either  Metaware  Development  or 
Agency  Model  related  tasks.  Each  is  listed  below,  along  with  the  researchers' 
names  and  organizations.  All  are  associated  with  companion  presentations 
(identifiable  by  combinations  of  title,  author  and  organization)  abstracted  in 
section  5. 


Metaware  Development 


MECOT"  Metaphor  Environment  Construction  Tools 

Nobuo  ASAHI,  Shoji  TANAKA  and  Akira  MAENAKA 
MITSUBISHI  ELECTRIC  CORPORATION 


Class  based  dynamic  user  interface  environment 
Masaya  UEDA 
SHARP  CORPORATION 


Browser  and  Examiner  of  Knowledge  Base  of  Visible  Software  Parts 
Toshihiko  SHIIROKAZE 
Oki  Electric  Industry  Co.,  Ltd. 


Sound  metaphors  in  3D  space 

Michio  MIWA,  Hiroyuki  HIKITA,  Hironobu  SUZUKI  and  Noritoshi 
OGASAWARA 

MATSUSHITA  ELECTRIC  INDUSTRIAL  CO.,  LTD. 


A  Metaphor  Environment  Based  on  Interactive  Scenarios 
Keiko  SAKURAGI  and  Fumitaka  KAWATE 
Sony  Corporation 


Visualizing  Information  by  Metaware 

Nobuyuki  MAKIMURA,  Tatsuya  KAINUMA  and  Makoto  MIZUYAMA 
FUJITSU  LIMITED 


Collaborative  Manipulation  Interface 

Takashi  SONODA  and  Mitsuhisa  KAMEI 
Fuji  Xerox  Co.,  Ltd. 


User  adaptation  by  a  task  intention  identification 
Kazuhisa  OKADA 
FRIEND21  Research  Center 


Agency  Model 


Synchronization  for  Accompaniment  System 
HIDAI  Yutaka 
FRIEND21  Research  Center 


Acoustic  Multimodal  Interaction  on  Accompaniment  System 
Takeshi  SUGAI 
FRIEND21  Research  Center 


On-Line  Handwriting  Recognition  Based  on  Agency  Model 

Koji  YURA,  Akinori  KAWAMURA  and  Tatsuya  HAY  AM  A 
TOSHIBA  CORPORATION 


Paint  It  Black  -  A  Multimodal  Drawing  Tool 

Mayumi  HIYOSHI,  Hideo  SHIMAZU,  Kenichi  OMACHI  and  Yosuke 

TAKASHIMA 

NEC  Corporation 


Video  Demonstration  of  a  prototype  system  for  retrieving  paintings 

Tadashi  SAITO,  Yoshinori  KAWAKAMl,  Akemi  KANEKO  and  Makoto 
NISHIDA 

DAINIPPON  PRINTING  CO.,  LTD. 


Information  structure  of  user  interface  for  integration  of  text  and  images 
Shogo  NAKAGOME  and  Motoichi  NAKAMURA 
TOPPAN  PRINTING  CO.,  LTD. 


Implementation  of  a  User  Adaptation  Mechanism  on  the  Agency  Model 
Shigeo  SUMINO  and  Mitsuru  IKEZAWA 
Hitachi,  Ltd. 


A  Three  Step  Filtering  Mechanism 
Masashi  UYAMA 
FRIEND21  Research  Center 


7.  Some  FRIEND21  Background  Papers 

Following  are  titles  of  some  English  language  FRIEND21  background  papers 
reviewed  during  the  preparation  of  this  report.  Some  are  FRIEND21  Research 
Center  reports  and  others  are  published  as  indicated.  All  were  provided  by  Mr. 
Kenji  Seno  of  P.I.E. 


FRIEND21  Project:  A  Construction  of  2U*  Century  Human  Interface 
Hajime  Nonogaki  and  Hirotada  Ueda 
FRIEND21  Research  Center 


FRIEND21  Project,  SIGCHI  Bulletin,  Volume  25,  Number  2,  April  1993 
Hirotada  Ueda 
FRIEND21  Research  Center 


A  Construction  of  Direct  Engagement  for  Human  Interface  and  Its  Prototyping, 
lEICE  Trans.  Fundamentals,  Vol.  E-75A,  No.  2,  February  1992 

Hajime  Nonogaki,  Norikazu  Saito,  Nobuo  Asahi  and  Makoto  Hirose 
(All  are  with  FRIEND21  Research  Center  except  Saito  who  is  with 
DAINIPPON  PRINTING  COMPANY,  LTD.) 


A  Construction  of  Direct  Engagement  for  2D*  Century  Human  Interface,  6*^^ 
Symposium  on  Human  Interface,  Oct.  24-26,  1990,  Tokyo 
Hajime  Nonogaki 
FRIEND21  Research  Center 


Strategy  for  Managing  Metaphor  Mismatching,  CHI  '92,  ACM  Conference  on 
Human  Factors  in  Computing  Systems,  Monterey,  California,  May  3-7,  1992. 
Makoto  Hirose 
FRIEND21  Research  Center 


A  Study  of  Personal  Adaptation  of  Metaware,  6***  Symposium  on  Human 
Interface,  Oct.  24-26,  1990,  Tokyo 

Nobuo  Asahi  and  Makoto  Hirose 
FRIEND21  Research  Center 


PLOTS:  An  interaction  paradigm  and  an  experiment  in  3D  shape  manipulation 
Michio  Miwa,  Takamasa  Oyama,  Miwa  Fukino,  Masao  Kato 
Tokyo  Research  laboratory,  Matsushita  Electric  Industrial  Co.,  Ltd. 


IMPACT:  An  Interactive  Natural-Motion-Picture  Dedicated  Multimedia 
Authoring  System 

Hirotada  Ueda,  Takafumi  Miyatake,  and  Satoshi  Yoshizawa 
Central  Research  laboratory,  Hitachi,  Ltd. 


Automatic  Structure  Visualization  for  Video  Editing 

Hirotada  Ueda,  Takafumi  Miyatake,  Shigeo  Sumino  and  Akio  Nagasaka 
Central  Research  laboratory,  Hitachi,  Ltd. 


Interactive  Video  Editing  Supported  by  Image  Recognition  Technology 
Hirotada  Ueda 

Central  Research  laboratory,  Hitachi,  Ltd. 


A  Blackboard-based  Architecture  for  Filtering  New  Software  Features 
Masashi  Uyama 
FRIEND21  Research  Center 


